Why It's Hard To Choose A VPN Provider

April 16, 2017

As @SwiftOnSecurity says:

“Just pick a good VPN” is like telling thirsty people to “go to a store and drink clear liquid.” They drank bleach, but at least you helped.

Yup, that pretty much covers it.

The amount of misinformation directed at potential VPN customers is vast. Worse, most prospective customers deeply misunderstand what VPNs do and under what circumstances they’re useful. It should be no surprise that this combustible mix of misinformation and misapprehension leads many astray.


Misinformation about VPNs comes in many forms and is not always easy to spot.

For example, a simple Google search for “best VPNappears to turn up reputable and timely VPN rankings. Alas, the top results are nearly all driven by affiliate marketing; placement in the rankings is entirely dependent on financial performance. It’s hard to imagine that the VPN with the best affiliate payouts is necessarily the most trustworthy and secure service on the market!

For a taste of how distasteful VPN affiliate marketing can be, peruse Google’s results for “VPN affiliate programs”. Here’s the current top hit; it gets worse from there:

If economic incentives are stacked against VPN consumers, so too it seems are the words of the providers themselves. The recent repeal of FCC privacy regulations led to a bonanza of new VPN customers and new VPN marketing efforts. Many providers conveniently forgot to mention important details about the repeal, including the fact that the privacy regulations in question had never gone into effect. If you didn’t feel the need to protect yourself from your ISP yesterday, perhaps you shouldn’t feel the need today. Moreover, when you use a VPN from home you’re effectively replacing your ISP with your VPN. Alas, the policies and behavior of VPN providers as a class is typically far less scrutinized than that of the largest ISPs. Is the tradeoff a clear win?

Publications with specific agendas also have a part to play in misleading customers about when VPNs are useful and what they’re useful for. For instance, TorrentFreak seems to think that third-party VPNs are useful for anonymity; in my opinion, they aren’t. Anonymity is a very strong condition that is hard to achieve in practice. Consider that people with deep technical skillsets and strong incentives to remain anonymous regularly fail to do so. I can’t decide if TorrentFreak’s language about VPNs is a sign of compromised ethics or just wishful thinking; either way, I’m sure it generates clicks.


Consumers interested in VPNs typically have zero experience with threat modeling. A good threat model addresses three questions:

  1. What information am I worried about having exposed?
  2. Who might be able to access that information?
  3. How might they be able access it?

The vast majority of customers can’t answer either of the first two questions with any clarity. Are you concerned about losing your banking password, being tracked by advertisers, or having interlopers learn about the sites you’re visiting? Is your attacker the NSA, your ISP, or people sitting in the same coffee shop? The right answers necessarily alter the strategy and, critically, the tools. (If your adversary is the NSA, no consumer VPN will help you. And if you’re worried about losing your banking password, breathe easy: your bank uses HTTPS everywhere; you don’t need a VPN, even on untrusted networks.)

Beyond this, the question of how an attacker might obtain one’s data requires a grasp of networking and software security that understandably goes far beyond the ken of all but the most sophisticated potential purchasers.

Needless to say, VPN providers take advantage of these fundamental misunderstandings to sell “enhanced privacy”, “true anonymity”, “military-grade encryption”, and other meaningless concepts. Snake oil sells.

Into this swirling chaos arrive the knights in shining armor, striving to shed light on a murky situation. Amongst them is That One Privacy Site (TOPS), which seems to have seen a massive spike in traffic after the United States’ ISP regulation repeal.

I have every reason to believe that the intentions of the TOPS website are good. Alas, even the white knights end up mostly leading consumers astray. TOPS attempts to provide objective metrics by which VPN providers can be judged. Information about providers is broken into categories like logging, data leaks, jurisdiction, etc. For truly savvy consumers, I suspect TOPS is extremely useful. For everyone else, I suspect it simply makes them miss the forest for the trees. Naive consumers visiting TOPS do the obvious thing: look for the VPN with the most green boxes and conclude that it’s the one to buy. The problem with this approach, as @KennWhite of the Open Crypto Audit project expressed to me:

Unlike trying to technically evaluate, say, IPSec vs OpenVPN which have fairly well-characterized objective security properties, assessing the security guarantees of a VPN service is less straightforward, and it basically comes down to trust signaling.

That strikes me as exactly right. When you use a VPN, you’re sending all of your data — presumably including unencrypted data — through your provider. No technical guarantees exist to ensure that your provider will be a trustworthy steward of this information. Instead, one must look at qualitative measures to evaluate a VPN; none of these are addressed by TOPS:

Questionable/sketchy product marketing & SEO, transparency of technical architecture decisions & assurances, openness to independent 3rd party review, ethical business practices, and — crucially — existence of an actual sustainable business model; these are all factors in evaluating the security posture.

Even along axes measured by TOPS, confusion can reign. For instance, privacy policies vary greatly amongst VPN providers. Plenty of reputable providers perform some kind of minimal time-limited logging both to help with customer support and to ensure that their network is responsive to abuse complaints. It’s absolutely possible for providers to both log data and provide meaningful security guarantees to customers. On the TOPS site, a “red” box in the policies or logging columns might genuinely be a red flag, or it might instead be the sign of a subtle and thoughtful provider.

Another axis that TOPS attempts to “measure” objectively is jurisdiction. With red boxes aplenty, naive consumers are quickly led to believe that VPN companies based in the United States or other Five Eyes countries are somehow worse on privacy matters. This conclusion is trivially wrong, of course. A VPN company based outside of Five Eyes can easily have a rotten privacy policy. Worse, they can simply fail to uphold the policy, silently abusing their customers’ trust. On the other hand, a VPN provider based in the United States is beholden to strict consumer protection laws and the potential scrutiny of agencies like the Federal Trade Commission. Policy documents should be legally binding; a good provider will have a government that enforces them.

Finally: TOPS is sometimes completely wrong. The TOPS maintainers have to keep a lot of information up-to-date so I don’t fault them for this. Nevertheless, the inaccuracies I’m aware of do make me suspicious of TOPS’ overall reliability. For example, TOPS claims that Cloak’s native apps leak IPv6 and DNS traffic. Neither is true. We’ve actually prevented leaks on dual-stack networks since our earliest alpha builds; we suspect this puts us ahead of the curve on such a fundamental security issue.

Trust signaling

In sum: it’s hard to choose a VPN wisely. For many potential customers, I worry it’s impossible.

Over time, I’ve developed a framework of six trust signals that I look for when I evaluate competing providers. I hope these will prove helpful in choosing your VPN provider:

  1. Do they have a clear and sensible threat model? (See Cloak’s; for more color, read my thoughts on security, privacy, and anonymity.)
  2. Do they have clear and unambiguous corporate ownership? (Cloak seems unique in offering actual pictures of our actual faces.)
  3. Do they have clear and unambiguous privacy policies? (See Cloak’s.)
  4. Do they have a demonstrable track record of security-mindedness and responsiveness? (See Cloak’s response to Heartbleed and to the “Looking Glass” paper.)
  5. Have they undergone a technical audit? (Cloak has and will again.)
  6. Do they charge money? (Free VPNs are usually the shadiest of the bunch, in my experience.)

It’s rarely easy to answer these questions! In my book, the VPN providers that make it easy to do so stand at the top of the heap.

Every once in a while, I think about starting “That Other One Privacy Site” to address these six broader concerns. Then I remember that I’m likely to simply become another pseudo white knight, contributing more to the noise than to the signal.

March 8, 2017 @ 04PM

Snap’s recent IPO convinces me there’s much more room for “innovation” on shareholder rights. SNAP shares are entirely non-voting and there’s simply no way for the interested public to get their hands on voting shares. That’s bold, but there’s so much more we can do!

For example, let’s IPO a company whose shares actively grant the company voting rights in the shareholder’s future personal decision making. I mean, I bet the whiz-kids at Snapchat haven’t thought of that!

Given how much SNAP shares are worth, it’s pretty clear that shareholder rights aren’t highly valued when the property is hot. I hope someone has the huevos rancheros to push it far.

March 3, 2017 @ 12PM

Marco Arment recently launched an ad network for Overcast, his podcasting app. I’ve always admired Marco’s willingness to experiment with new business models. I suspect (and hope!) that this proves to be a gold mine. I also suspect that, somewhere down the road, it will prove worthwhile for Marco to implement an auction-based pricing model.

February 15, 2017 @ 02PM

Lawfare posted an interesting deep-dive on the law of leaks. The introduction discusses Trump’s recent tweet that the “real” story is about illegal leaks:

First, the President makes these accusations despite not knowing the actual source of these leaks. At least some of the information seems to be coming from his own White House. And nothing that has come to light is the kind of material that only the FBI or NSA would be aware of. Indeed, there is no particular reason to assume that any of these leaks are intelligence community leaks, rather than leaks by current and former White House officials with the knives out for Flynn.

Second, these tweets suggest that the President is more interested in hunting down leakers than in getting to the bottom of extremely serious allegations against his own administration. Whether Trump’s comments represent an intentional deflection or merely reflect misaligned priorities, most people can agree without defending leaking that the leaks are probably not the “real scandal” here.

Finally, and perhaps most worryingly, the President’s statement seems to signal an intention to use the pretense of leak investigations to engage in political retaliation. As Tim Edgar noted yesterday, the President is showing an instinct here that is not all that dissimilar from the events that set Watergate in motion.

February 13, 2017 @ 03PM

From Phil Klay’s What We’re Fighting For:

If we choose to believe in a morally diminished America, an America that pursues its narrow selfish interests and no more, we can take that course and see how far it gets us. But if we choose to believe that America is not just a set of borders, but a set of principles, we need to act accordingly. That is the only way we ensure that our founding document, and the principles embedded within, are alive enough, and honorable enough, to be worth fighting for.

February 2, 2017 @ 05PM

With apologies for the extreme degree of meta.

If you’re one of the three or four people who follows my blog, you’ve probably noticed that I’ve been posting daily for the past full month. That’s a first in my blog’s 15+ year history.

I wanted to do this in part because the four sections of my site — blog, microblog, photoblog, and audioblog — are finally well-separated. Each has its own landing page and RSS feed; my home page aggregates all content and offers a feed to match.

Today’s post caps off the full month with an audioblog post. Historically I’ve also transcribed my audio posts, but that’s probably wasted effort. Going forward, I plan to pair my audio with a small amount of explanatory text.