Why It's Hard To Choose A VPN Provider

As @SwiftOnSecurity says:

“Just pick a good VPN” is like telling thirsty people to “go to a store and drink clear liquid.” They drank bleach, but at least you helped.

Yup, that pretty much covers it.

The amount of misinformation directed at potential VPN customers is vast. Worse, most prospective customers deeply misunderstand what VPNs do and under what circumstances they’re useful. It should be no surprise that this combustible mix of misinformation and misapprehension leads many astray.

Misinformation

Misinformation about VPNs comes in many forms and is not always easy to spot.

For example, a simple Google search for “best VPN” appears to turn up reputable and timely VPN rankings. Alas, the top results are nearly all driven by affiliate marketing; placement in the rankings is entirely dependent on financial performance. It’s hard to imagine that the VPN with the best affiliate payouts is necessarily the most trustworthy and secure service on the market!

For a taste of how distasteful VPN affiliate marketing can be, peruse Google’s results for “VPN affiliate programs”. Here’s the current top hit; it gets worse from there:

If economic incentives are stacked against VPN consumers, so too it seems are the words of the providers themselves. The recent repeal of FCC privacy regulations led to a bonanza of new VPN customers and new VPN marketing efforts. Many providers conveniently forgot to mention important details about the repeal, including the fact that the privacy regulations in question had never gone into effect. If you didn’t feel the need to protect yourself from your ISP yesterday, perhaps you shouldn’t feel the need today. Moreover, when you use a VPN from home you’re effectively replacing your ISP with your VPN. Alas, the policies and behavior of VPN providers as a class is typically far less scrutinized than that of the largest ISPs. Is the tradeoff a clear win?

Publications with specific agendas also have a part to play in misleading customers about when VPNs are useful and what they’re useful for. For instance, TorrentFreak seems to think that third-party VPNs are useful for anonymity; in my opinion, they aren’t. Anonymity is a very strong condition that is hard to achieve in practice. Consider that people with deep technical skillsets and strong incentives to remain anonymous regularly fail to do so. I can’t decide if TorrentFreak’s language about VPNs is a sign of compromised ethics or just wishful thinking; either way, I’m sure it generates clicks.

Misapprehension

Consumers interested in VPNs typically have zero experience with threat modeling. A good threat model addresses three questions:

1. What information am I worried about having exposed?
2. Who might be able to access that information?
3. How might they be able access it?

The vast majority of customers can’t answer either of the first two questions with any clarity. Are you concerned about losing your banking password, being tracked by advertisers, or having interlopers learn about the sites you’re visiting? Is your attacker the NSA, your ISP, or people sitting in the same coffee shop? The right answers necessarily alter the strategy and, critically, the tools. (If your adversary is the NSA, no consumer VPN will help you. And if you’re worried about losing your banking password, breathe easy: your bank uses HTTPS everywhere; you don’t need a VPN, even on untrusted networks.)

Beyond this, the question of how an attacker might obtain one’s data requires a grasp of networking and software security that understandably goes far beyond the ken of all but the most sophisticated potential purchasers.

Needless to say, VPN providers take advantage of these fundamental misunderstandings to sell “enhanced privacy”, “true anonymity”, “military-grade encryption”, and other meaningless concepts. Snake oil sells.

Into this swirling chaos arrive the knights in shining armor, striving to shed light on a murky situation. Amongst them is That One Privacy Site (TOPS), which seems to have seen a massive spike in traffic after the United States’ ISP regulation repeal.

I have every reason to believe that the intentions of the TOPS website are good. Alas, even the white knights end up mostly leading consumers astray. TOPS attempts to provide objective metrics by which VPN providers can be judged. Information about providers is broken into categories like logging, data leaks, jurisdiction, etc. For truly savvy consumers, I suspect TOPS is extremely useful. For everyone else, I suspect it simply makes them miss the forest for the trees. Naive consumers visiting TOPS do the obvious thing: look for the VPN with the most green boxes and conclude that it’s the one to buy. The problem with this approach, as @KennWhite of the Open Crypto Audit project expressed to me:

Unlike trying to technically evaluate, say, IPSec vs OpenVPN which have fairly well-characterized objective security properties, assessing the security guarantees of a VPN service is less straightforward, and it basically comes down to trust signaling.

That strikes me as exactly right. When you use a VPN, you’re sending all of your data — presumably including unencrypted data — through your provider. No technical guarantees exist to ensure that your provider will be a trustworthy steward of this information. Instead, one must look at qualitative measures to evaluate a VPN; none of these are addressed by TOPS:

Questionable/sketchy product marketing & SEO, transparency of technical architecture decisions & assurances, openness to independent 3rd party review, ethical business practices, and — crucially — existence of an actual sustainable business model; these are all factors in evaluating the security posture.

Even along axes measured by TOPS, confusion can reign. For instance, privacy policies vary greatly amongst VPN providers. Plenty of reputable providers perform some kind of minimal time-limited logging both to help with customer support and to ensure that their network is responsive to abuse complaints. It’s absolutely possible for providers to both log data and provide meaningful security guarantees to customers. On the TOPS site, a “red” box in the policies or logging columns might genuinely be a red flag, or it might instead be the sign of a subtle and thoughtful provider.

Another axis that TOPS attempts to “measure” objectively is jurisdiction. With red boxes aplenty, naive consumers are quickly led to believe that VPN companies based in the United States or other Five Eyes countries are somehow worse on privacy matters. This conclusion is trivially wrong, of course. A VPN company based outside of Five Eyes can easily have a rotten privacy policy. Worse, they can simply fail to uphold the policy, silently abusing their customers’ trust. On the other hand, a VPN provider based in the United States is beholden to strict consumer protection laws and the potential scrutiny of agencies like the Federal Trade Commission. Policy documents should be legally binding; a good provider will have a government that enforces them.

Finally: TOPS is sometimes completely wrong. The TOPS maintainers have to keep a lot of information up-to-date so I don’t fault them for this. Nevertheless, the inaccuracies I’m aware of do make me suspicious of TOPS’ overall reliability. For example, TOPS claims that Cloak’s native apps leak IPv6 and DNS traffic. Neither is true. We’ve actually prevented leaks on dual-stack networks since our earliest alpha builds; we suspect this puts us ahead of the curve on such a fundamental security issue.

Trust signaling

In sum: it’s hard to choose a VPN wisely. For many potential customers, I worry it’s impossible.

Over time, I’ve developed a framework of six trust signals that I look for when I evaluate competing providers. I hope these will prove helpful in choosing your VPN provider:

1. Do they have a clear and sensible threat model? (See Cloak’s; for more color, read my thoughts on security, privacy, and anonymity.)
2. Do they have clear and unambiguous corporate ownership? (Cloak seems unique in offering actual pictures of our actual faces.)
3. Do they have clear and unambiguous privacy policies? (See Cloak’s.)
4. Do they have a demonstrable track record of security-mindedness and responsiveness? (See Cloak’s response to Heartbleed and to the “Looking Glass” paper.)
5. Have they undergone a technical audit? (Cloak has and will again.)
6. Do they charge money? (Free VPNs are usually the shadiest of the bunch, in my experience.)

It’s rarely easy to answer these questions! In my book, the VPN providers that make it easy to do so stand at the top of the heap.

Every once in a while, I think about starting “That Other One Privacy Site” to address these six broader concerns. Then I remember that I’m likely to simply become another pseudo white knight, contributing more to the noise than to the signal.