The Great Firewall of China is Not so Great, Afterall

We crack open the technology that keeps 1.4 BILLION people walled in

Most people are familiar with China’s widespread internet censorship. As VPNs are blocked there, it’s hard for people in China to access the “outside world” online.

In this article, I share my personal experiences of trying to run a VPN service for users in China, as well as the general quality and speed of the internet there.

This is the first in a series of articles that dive deeper into the issues faced by internet users living in digitally oppressed regimes.

Is the Great Firewall of China a Great Myth?

We first launched our flagship desktop and mobile applications MysteriumVPN in 2018. The VPN was available to our Chinese users for 2 years following our launch.

This led me to the following conclusions:

The Great Firewall (GFW), deep packet inspection and "learn, filter and block" for OpenVPN, UDP, or other restricted services don't really exist. Or, at least, they are not as sophisticated as we've been led to believe.

Perhaps the reputation and mystery of the Great Firewall have been overestimated. Developers like to talk about it extensively, as it’s an interesting challenge.

If it’s the second point, the topic is likely wrapped up in a lot of rumors. This matters for our team as we build new, anti-censorship tech from scratch.

In fact, Mysterium Network p2p VPN builds across several emerging technologies. This meant that we needed to prioritise early on in our development.

We have been focused on bringing peer to peer payments into Mysterium Network as a core focus. It’s easy to get caught up in rabbit holes online — DHT & Kademlia, obfuscated transports — which is not ideal while building a VPN startup. We didn’t place too much focus on fancy networking features, so as to avoid premature optimisation.

We stuck to a simple solution — OpenVPN server-client and REST APIs. This worked fine for our Chinese users, for more than a year.

Image: VPN throughput for China users

Until one day I noticed a big drop in Mysterium Network Testnet health metrics:

Image: Simultaneous sessions from China

Making requests from China

What happened? And how to fix (debug) this? To find out, I first needed a way to reproduce the VPN connection from China. The tool ping.pe came in handy:

Image: Our domain records are being black-holed to unreachable machines

Here we have a window into how the GFW works differently from the regular internet.

While the rest of the world follows a standard practice when it comes to how the internet works, China has decided to create its own standard. 😅

The “Great Firewall” as we know it is causing the DNS server to return an incorrect IP address for Mysterium’s domain [https://testnet.mysterium.network/], which results in traffic being diverted & black-holed to unreachable machines.

This technique is referred to as DNS interference, DNS poisoning or DNS spoofing.

Verify the blocking technique

So, I thought, let’s try to bypass DNS altogether and connect to our precious API via the IP address directly:

Image: The server is actually reachable via IP address

From this, the blocking technique of the GFW is clear — it is DNS poisoning and black-holing. It seems actual traffic can pass through our datacenter in Berlin.

Conclusion: When you are in China you can’t trust DNS responses.

So, we know how to unblock ourselves — by bypassing the DNS altogether.

This creates clear steps for the Mysterium development team to be able to offer VPN service in China. All I have to develop is a feature to bypass a DNS...

Packet loss is 56%. Seriously?

But still, I noticed — why did one of the requests from my previous debugging fail (Jiangsu → to Berlin)?

IMHO, there’s nothing *wrong* here. It’s actually the quality of the Internet itself. So I checked, by pinging this server:

Image: Quality of connectivity from Germany to various locations

Turns out my guess was right. While most of the world has good Internet connectivity to all locations, the exception is China, which has a packet loss of 56% — seriously?

I can’t even imagine how people are using such a slow service in our world of the “9-Second attention span”.

In my opinion, good Internet transport is important. It provides fast transactions for people and businesses, and enables overall economic growth. This is relevant across all public infrastructure — roads, railroads, ports — and the internet too.

It is time for us to recognise that the internet is public infrastructure.

Why were the VPN APIs targeted and blocked by GFW?

So why was Mysterium VPN targeted? Actually, it was not necessarily the VPN that was singled out. The DNS zone *.mysterium.network, together with VPN APIs, were all black-holed. This was due to the naming convention of our VPN APIs (i.e. using mysterium.network subdomains) more so than any fancy blocking technique.

At this point in time, our communications strategy had turned more political. My hypothesis is that this was the cause of our VPN service being temporarily banned in China. (The good news, we’ll be back up and running soon.)

Examples of our content:

An opinion piece on the coronavirus cover-up: a closer look at internet censorship in China
A general overview of the centralized vs. decentralized Internet, and why censorship sucks
Geoblocking and its role in politics and economics
Tor vs. VPN — what’s the difference?

It seems our content was picked up by Chinese censors. Then China got mad at us for sharing these opinions, so they blocked us all together.

What’s up with your internet China?

It might be that the Great Firewall of China is not so great. They censor sites for sure, but when it comes to sophisticated deep packet inspection, it might be that they just degrade the quality of service.

Wikipedia article on GFW blocking methods somewhat confirms this:

Quality of service filtering — Since 2012, the GFW is able to “learn, filter and block” users based on traffic behavior, using deep packet inspection.[47] This method was originally developed for blocking VPNs..

So, what’s up with your internet China?

I will be sharing share my journey on unblocking our VPN connection for China. Stay tuned on Twitter @valdas_da_coder— in the next article I will cover the challenges of buying a hosting server in Mainland China and how I tested the quality of service there.