Ivan Ristić

My Infosecurity London 2012 SSL Panel Notes

Last month, GlobalSign invited me to participate on an SSL Panel at Infosecurity London. Other participants were David Holmes (F5), Ryan Hurst (GlobalSign), and Steve Roylance (GlobalSign, moderator). These are my rough notes. I hope you will find them interesting.

Can you tell us about yourself and why you’re here?

In 2009 I quit my job (because I had become bored), and I decided to work on something I can be passionate about. As it happens, that something was SSL. I had discovered SSL a few years before, while working on my book Apache Security, and always wanted to go back to it, to research it in more detail. To my dismay, I discovered that SSL, arguably the most important security protocol on the planet, is poorly documented, difficult to configure correctly, and that there are no tools to test SSL configurations. That's when SSL Labs was born. Today, SSL Labs offers the best assessment platform and for free. In the past 2 years we've conducted several large-scale surveys of public SSL implementation in an effort to understand exactly where we are, when it comes to the security of the Web.

Talk to us about your perspective on the last two years and the breaches at third-party trust providers?

I think that, in the last 2 years we learned that security is not static. It's not something you work on, and then leave it behind. You know, SSL was designed in 1994, and, today, in 2012, remains essentially the same, conceptually. In the meantime, we saw the Web evolve, and evolve around SSL. The threat model of today is vastly different to the threat model of 1994. Eighteen years! So we dropped the ball. But I think that's actually how things work, for us humans. We don't think about security while it's not a problem. As a result of that, security it becomes a real problem eventually, and then we start to think about it. It's a cycle; a never-ending cycle.

How would you describe the current situation of Public Trust on the Internet?

It's not as bad as it seems. I think that, practically speaking, our biggest problem is not that we have too many CAs, but that we cannot yet build secure systems. The events from the last two years made us realise that any system can be broken into, and -- surprise, surprise -- even the CAs. Once you accept that, the obvious problem with Public Trust is that any one CA can create a certificate for any web site. Site owner's permission is not even required. I am hopeful that that will be fixed with a new standard  supporting public key pinning. Then, site owners will be able to say which CAs can issue certificates for their web sites.

Why aren’t Extended Validation certificates saving us now?

You know, back in the day, all certificates used to be "EV". I remember very well jumping through many hoops to get a certificate -- so long ago I don't even remember what year it was. I don't think that many people realise that, prior to EV certs, there wasn't a standard for certificate issuance. Today, there isn't a standard for non-EV certificae issuance [we may get one officially any time now]. So I welcome EV certificates, because they connect your online presence to your off-line identity. That's great, if you need it or if you want it. At the same time, DV certificates are getting cheaper, which is how it should be. The basic security of the communication channel should be available to everyone.

What is your take on the recent advances in cryptographic and protocol focused attacks?

The recent attacks against SSL are a sign of protocol maturity. The reality is that we are not smart enough to foresee all problems when we're designing protocols. So the only way to arrive at something that is secure, is to give it our best shot, start to use it, and fix it as problems are discovered. The recent attacks against SSL demonstrate that people are looking at SSL, and that's a great thing. That means that we're improving. The sky is not falling; but please don't tell everyone. I don't mind the sensational headlines, because they get people's attention. Without the headlines -- without fear -- we wouldn't be able to improve security in the same way. For example, about 75% of all SSL sites are vulnerable to the BEAST attack. I wouldn't mind seeing a sensationalist headline scaring everyone to improve their configuration. The reality is that we need more headlines and more working exploits. How CISO’s should approach trust providers and risk? Choose a CA that you can trust, and a company for which certificate issuance is a significant business (revenue stream). Most of the risk is in-house, in which how you manage your certificates, configure your SSL servers, and implement your applications.

If you were to sit down with an IT manager today and talk to them about his biggest risks relating to SSL what would you tell them?

The biggest SSL-related risk is at the application layer. Your SSL server may be misconfigured today, but you can fix that very quickly. We have people testing themselves using our online assessment tool, getting a bad grade, improving their configuration and even getting a new certificate, just an hour later. And, to be honest, no one is attacking SSL directly. The real risk lies in the application layer, when applications use features that completely subvert SSL. It's like SSL isn't there. We conducted a deep survey of these issues last yar (in the most popular web sites), and the conclusion was that most sites are vulnerable, and SSL effectively useless. I have a lot of hope for declarative security measures. For example, HTTP Strict Transport Security is an emerging standard where you can declare that your application/site uses only SSL (never plain-text HTTP). That means that, even if your application contains implementation errors, you remain secure. I also have a lot of hope for new protocols, which will include SSL as a mandatory component. Eventually, we will migrate to an Internet that is 100% encrypted. I hope.

What do you think is the future of Public Trust?

Realistically speaking, I think we will stay pretty much where we are. Public Key Pinning will remove the most obvious problem. To change anything else -- that's too much work for anyone's taste. I would very much like to see browsers incorporate some elements of crowd-source (Covergence-style) trust for those who understand it. Password-based SSL authentication could be interesting.

What changes are being proposed to augment existing security technologies and how these changes may affect the industry?

We've seen many proposals, but not all of them are equally easy to implement. For example, there's DNSSEC that implements a secure DNS, and DANE, which is a bridge between secure DNS and PKI. Because there is no doubt that we need a secure DNS, I have no doubt that DNSSEC will be widely deployed, eventually (it will take a few years). In the meantime, low-effort proposals, such as HSTS and Public Key Pinning, are likely to become practical. Technically speaking, other proposals can become reality, too, but they require a lot of lobbying and involve a lot of politics.

What do you think about DNSSEC and DANE?

There's one aspect of DANE that I am not sure about, and that's the ability to have valid certificates without CAs. It's not that I like CAs very much, but we forget that the PKI infrastructure was designed to be independent of DNS. And, with DNSSEC, we will revert to only one trust root -- that in the DNS. On the other hand, everyone deserves basic security, and that's something DNSSEC will most certainly achieve. The biggest problem is that with current CA-based ecosystem or with DNSSEC, site owners have no say.

What are the biggest problems of the security industry?

Our biggest problem is in having the security industry in the first place. Security is the business of the developers (used here in a wider sense, to mean those who implement things.) We have the security industry today only because those who are implementing our systems are not building security in. And that's our biggest problem. It's only when the economics of secure development improve that we are going to see substantial improvement in security. Apart from that, our biggest problem is complacency. For example, the CAs, who were (and are) getting serious money from issuing certificates, could have stayed on top of things. The could have tracked the threat model and reacted to new threats. (I don't actually blame CAs for that. I mean, I do, but, ultimately, a wider community should take the responsibility.) Browser vendors could have fixed usability issues (rather than putting the burden of security on the shoulders of end users). And so on... We need our incentives aligned, so that it is in everyone's interest to have better security.

Announcing SSL Pulse

Last week we announced SSL Pulse, a continuously updated dashboard that is designed to show the state of the SSL ecosystem at a glance. While it is possible today to deploy SSL and to deploy it well, the process is difficult: the default settings are wrong, the documentation is lacking, and the diagnostic tools are inadequate. For these reasons, we cannot say that the Web is yet secure, but we hope that someday it will be. The purpose of SSL Pulse is to bring visibility to SSL implementation issues on the Web, and while businesses are starting to fix these issues we can keep track of progress made towards making SSL more robust and widely adopted on the Internet.

SSL Pulse is based on the assessment technology and testing conducted by SSL Labs. The underlying data set draws from the information on about 200,000 SSL web sites that represent the most popular web sites in the world. We cherry-picked only the most important data points, focusing especially on those aspects where improvements are needed. We have so far conducted only one round of testing, but, when the next month’s results become available, we will start to show historic values and hopefully see improvements for each data point.

So what do the results tell us? Looking at the SSL Labs grades, which are designed to sum up the quality of SSL configuration, we can see that about 50% (99,903 sites) got an A, which is a good result. Previous global SSL Labs surveys reported about 33% well-configured sites, which means that more popular sites are better configured. Unfortunately, many of these A-grade sites (still) support insecure renegotiation (8,522 sites, or 8.5% of the well-configured ones) or are vulnerable to the BEAST attack (72,357 sites, or 72.4% of the well-configured ones). This leaves us with only 19,024 sites (or 9.59% of all sites) that are genuinely secure at this level of analysis.

The number of sites vulnerable to insecure renegotiation is decreasing at a steady pace, as patches are applied or servers get replaced. The very high number of sites vulnerable to the BEAST attack is worrying, because this problem needs to be addressed in configuration, and that requires awareness, time, and knowledge. Plus, freshly installed systems are equally likely to be vulnerable because of the insecure defaults.

Among other interesting data points, we found only 19 weak private keys in our data. There are also 9 keys that trigger our black list of weak Debian keys. The support for HTTP Strict Transport Security, which is the state of the art configuration for SSL, is at 0.85% (1,697 sites).

As part of this effort, we also published an SSL/TLS Deployment Best Practices guide with clear and concise instructions to help overworked administrators and programmers spend the minimum time possible to deploy a secure site or web application.

This post was originally published on the Trustworthy Internet Movement blog.

Qualys supports reform at CA/Browser Forum

Last month, the CA/Browser Forum announced the creation of a working group that will focus on organizational reform. We welcome the announcement with open arms; the public PKI infrastructure needs more structure, collaboration, and visibility, and the CA/Browser Forum is in the best position to advance the robustness of the infrastructure in the short term.

The PKI infrastructure has been evolving organically for too long, and, because of that, we are today faced with many of its structural cracks. The challenge now for the security community (not just the CA/Browser Forum) is to:

• understand the complexities of the public PKI infrastructure,
• bring all the key stakeholders together,
• establish a system of governance that serves the collective interests,
• realign stakeholders' incentives to make the ecosystem stronger,
• in the short-term, resolve key structural and technical issues,
• maintain an up-to-date threat model and address weaknesses proactively, and,
• in the long-term, evolve the ecosystem to adequately address the real threats.

There is no doubt that a reform is needed; what is not clear is how it will take place. To facilitate the change, the CA/Browser Forum will need to transform substantially, inviting a wide participation as well as embracing openness and transparency. There is already a large number of organizations and individuals working on improving the security of the PKI infrastructure; those efforts need to be streamlined, and the changes orchestrated.

Some of the pressing issues that need to be addressed include the following:

• The large attack surface stemming from a compromise of any one Certificate Authority
• Not enough visibility into the operation of Certificate Authorities
• Insufficiently defined operational requirements and auditing standards
• Lack of reliable control mechanisms and ability to deal with failures
• Low adoption rate of SSL/TLS across all web sites
• Numerous configuration and implementation issues that subvert security in those sites that did adopt SSL/TLS
• Inadequate browser SSL/TLS implementations that do not make security seamless and easy (instead pushing the burden of security onto the shoulders of the end users, who are not in the position to make informed decisions), but still make it difficult for advanced users (who are in the position to make informed decisions) to pursue alternative approaches

This post was originally published on the Qualys Security Labs blog.

SSL and Browsers: The Pillars of Broken Security

Last week at the RSA Conference, Wolfgang and I gave a talk called SSL and Browsers: The Pillars of Broken Security. We could have easily called it a State of SSL, because we went through the most relevant SSL issues today, demonstrated the problems using the data from SSL Labs surveys, and discuss how the issues can be overcome. Here's the talk summary:

Recent attacks on browsers and certificate authorities for SSL have shown how fragile these systems are, yet we all depend on them while using the Internet on daily basis. This talk will explore the implementation flaws in the SSL protocol and the browsers that support it. The speakers will showcase extensive research collected from millions of websites that reveal the state of SSL and Browse Security on the Internet. The session will then explore the mitigation options for the problems we are experiencing today, and provide a framework in which we can solve future SSL security issues.

Get this slides here:

• SSL and Browsers: The Pillars of Broken Security (PDF, 4 MB)

This post was originally published on the Qualys Security Labs blog.

Announcing the SSL/TLS Deployment Best Practices guide

SSL/TLS is a deceptively simple technology. It is easy to deploy, and it just works . . . except that it does not, really. The first part is true—SSL is easy to deploy—but it turns out that it is not easy to deploy correctly. To ensure that SSL provides the necessary security, users must put more effort into properly configuring their servers.

In 2009, we began our work on SSL Labs because we wanted to understand how SSL was used and to remedy the lack of easy-to-use SSL tools and documentation. We have achieved some of our goals through our global surveys of SSL usage, as well as the online assessment tool, but the lack of documentation is still evident. This document is a first step toward addressing that problem.

Our aim here is to provide clear and concise instructions to help overworked administrators and programmers spend the minimum time possible to obtain a secure site or web application. In pursue of clarity, we sacrifice completeness, foregoing certain advanced topics. The focus is on advice that is practical and easy to understand. For those interested in advanced topics, we provide references at the end of the guide.

Download the guide:

• SSL/TLS Deployment Best Practices (PDF, 500 KB)

This post was originally published on the Qualys Security Labs blog.

IronBee reboot

We announced IronBee, our next-generation web application firewall engine, exactly one year ago at the RSA Conference in San Francisco. Although we had expected to have a fully working product by now, it did not happen. In this post I will explain why.

This time last year, we had a core of the product built on top of libhtp, our security-aware HTTP parsing library. We had also put the project infrastructure out in the public, on GitHub and SourceForge. The initial public release was 0.2, and the state of the project was pretty much what you would expect (except for libhtp, which had been developed earlier and was pretty decent). The goal of the early announcement was to get the word out and hopefully get interested parties to join us. It didn't work. The feedback was overwhelmingly positive and many were genuinely interested in working on IronBee, but, at the end of the day, no one followed through.

The lack of contributors did not stall our project. What did stall it was the fact that our entire development team was busy with other projects, and our inability to hire a full-time developer for the project. It was not until November that we had hired Nicholas LeRoy to fill the developer role, and that's when the development started to pick up. We were also able to free additional resources for work on IronBee.

The official reboot happened internally about a month ago, but we are making it public now. This year's RSA Conference is next week, and we're aware that people will be asking questions about our progress. If it weren't for that, we would probably keep quiet for another month or so, until we had more to show.

In the following weeks, we will start to make regular releases, improve the documentation, and start to write here about our progress and, especially, the new interesting features we have in IronBee. Finally, we will start to track our progress on the development roadmap.

This post was originally published on the IronBee Blog.

TLS Renegotiation and Denial of Service Attacks

A group of hackers known as THC (The Hacker's Choice) last week released an interesting DoS tool that works at the SSL/TLS layer. The tool is exploiting the fact that, when a new SSL connection is being negotiated, the server will typically spend significantly more CPU resources than the client. Thus, if you are requesting many new SSL connections per second, you may end up using all of the server's CPU.

The issue abused by the tool is not new, but what is new is that we now have a well publicised working exploit, and that always makes you pay attention. In addition, the tool uses the renegotiation feature, which means that it can force a server to perform many expensive cryptographic operations over a single TCP connection. It's not clear if the relying on renegotiation helps with the DoS attack (there's a very good analysis of the trade-offs on Eric Rescorla's blog), however the fact that external DoS mitigation tools (e.g., rate limiting setups) are only seeing one TCP connection certainly helps with avoiding detection.

But that's only if your server supports client-initiated renegotiation. If it does not, anyone wishing to perform a DoS attack against the SSL layer will have to fall back to using one TCP connection for one SSL connection. IIS, for example, does not support client-initiated renegotiation. Apache used to, but changed its behaviour when implementing RFC 5746 (which fixed the TLS Authentication Gap problem). Even if you depend on a product that does support client-initiated renegotiation, chances are you can easily disable that feature. And, when you do, you are not going to miss it (unlike server-initiated renegotiation, which some sites that require client certificates might need).

To help you with assessing your systems for this weakness, we have updated the SSL Labs assessment tool to test not only if secure renegotiation is supported (which we've been testing for some time now), but also to check if secure client-initiated renegotiation is enabled. Previously we only tested for insecure client-initiated renegotiation.

The sensible thing to do is to check for client-initiated renegotiation support in your servers, and disable it where possible. Although that won't substantially help you overall (defending against DoS attacks is notoriously difficult and expensive), it will harden your defences against this particular technique.

This post was originally published on the Qualys Security Labs blog.

Mitigating the BEAST attack on TLS

During the summer rumours about a new attack against SSL started circulating. Then Opera released a patch, but made no comment about what it was patching. Eventually enough information leaked out that some smart people figured what the attack was about. What remained unknown was the exact technique used in the proof of concept, and that was eventually explained in Thai's blog post. For a comprehensive overview of related links, go to Thierry Zoller's blog post on BEAST.

As it turns out, the attack itself was conceived years ago, deemed impractical, but it was nevertheless fixed in TLS 1.1. The new attack technique introduced a few optimizations to make it practical.

In terms of mitigation, I expect this problem will be largely addressed on the client side, despite a potential compatibility problem that may cause some TLS sites to stop working. The only reliable way to defend against BEAST is to prioritise RC4 cipher suites, as proposed by PhoneFactor.

Just as an example, here's one way to do the above in Apache:

SSLHonorCipherOrder On
SSLCipherSuite RC4-SHA:HIGH:!ADH

Not everyone likes RC4, even though there is little to no evidence that it is insecure in the context of SSL/TLS. If your server supports TLS 1.2+ you can try the approach recommended by Steve Caligo:

SSLHonorCipherOrder On
SSLCipherSuite ECDHE-RSA-AES128-SHA256:AES128-GCM-SHA256:RC4:HIGH:!MD5:!aNULL:!EDH

The idea is that you put a few TLS 1.2 cipher suites first so that they can be picked up by TLS 1.2 clients, which are not vulnerable, followed by RC4 for TLS 1.0 clients.

Now that I've discussed what works as mitigation, let's look at a few approaches that do not work:

• Supporting TLS 1.1+ server-side is a good start, but does not amount to much because very few clients support newer versions of the protocol at this time. And even with TLS 1.1+ support client-side, there's nothing preventing the MITM to force a protocol downgrade back to TLS 1.0. (For a discussion on defense techniques against downgrade attacks, see this thread on the TLS WG mailing list).
• Enabling the empty fragment technique server-side (details for OpenSSL here) does not work either. TLS 1.0 uses two initialisation vectors (IVs), one each for client- and server-side of the communication channel. The vulnerability exploited by BEAST is on the client-side and cannot be addressed by making server-side changes to how data is sent.
• Compression is said to make the attack impossible, but, as with TLS 1.1+, the support for it client-side is inconsistent.

Update (20 Jan 2012): In testing OpenSSL 1.0.1-beta2, which came out yesterday, I realised that it will happily negotiate AES-CBC-SHA256 even on a TLSv1.0 connection. So I removed it from the recommendation, replacing it with two other TLSv1.2 cipher suites.

This post was originally published on the Qualys Security Labs blog.

SSL Labs: Announcing launch of two Convergence notaries

Convergence is Moxie Marlinspike's attempt to introduce fresh thinking into the debate about PKI, certificate authorities, and trust. A hint of what was in the works was in a blog post published in April (SSL And The Future Of Authenticity); the project was launched at Black Hat US in August. Moxie's talk (here's the video on YouTube) was entertaining and insightful.

Moxie advertises the project as a way of dispensing with certificate authorities ("An agile, distributed, and secure strategy for replacing Certificate Authorities"). At the first glance that's true. You get a browser add-on (only Firefox for the time being) that, once activated, completely replaces the existing CA infrastructure. Whenever you visit an SSL site your browser will talk to two or more remote parties (notaries) and ask them to check the site's certificate for you. If they both see the same certificate you decide to trust the site.

But when you dig deeper into the project, you realise that it consists of two parts. The first, and more important, part is the ability to delegate trust decisions from your browser to another party that's remote to you. That means that you are no longer forced to accept the decisions of the browser vendors, but you can make your own. That ability is, for me, the most thrilling aspect of the project.

The second part of the project is the current backend implementation that makes trust decisions. The approach is great in its simplicity: if you can see the same certificate from several different locations you conclude that it must be the correct certificate. We mustn't rush, however. We've just been given the ability to choose whom to trust, and it's too soon to settle on any one implementation. I am far more interested in experimenting with different approaches, to see what works and what does not.

To that end, it makes me very happy to announce that we (Qualys) have decided to support Convergence by financing and running two notary servers. While it's not yet clear if Convergence can succeed (there are many technological and adoption challenges to conquer), we want to play a part in it and help it succeed.

Finally, here are the links to the notary servers (one of which is in the US and the other in Europe):

• https://www.ssllabs.com/convergence/notary-eu.convergence.qualys.com.notary
• https://www.ssllabs.com/convergence/notary-us.convergence.qualys.com.notary

Note: To use the above links, you have to have the Convergence plugin installed. After that, all you need to do is click on the links and the notaries will become part of your configuration. Please report any problems to convergence-notary@qualys domain name.

Key SSL/TLS mailing lists to follow

The best way to really learn about SSL/TLS and related topics is to follow the discussions on the key technology mailing lists. These lists are always interesting, but especially so in turbulent times, when the value of participation simply goes through the roof.

The key mailing lists are the following:

• Transport Layer Security Working Group (core protocols); list archive
• Web Security Working Group (HSTS); list archive
• SSL Observatory (trust ecosystem); list archive

The list archives are publicly available, so there's not even a need to subscribe.