Fexyn
Fexyn
All posts

How does a VPN work? A clear technical explanation

Fexyn Team··16 min read

How does a VPN work? Your device builds an encrypted tunnel to a server you choose, sends every packet through it, and that server forwards your traffic to the real destination using its own IP. To anyone watching the network between you and the VPN server, all they see is encrypted noise heading to one address. To anyone watching past the VPN server, your traffic looks like it came from the server, not from you.

That is the 30-second version. The rest of this post unpacks each step with the real protocol detail, because "encrypted tunnel" is a phrase that hides a lot of interesting machinery. We will cover what actually happens during a VPN handshake, what encapsulation means at the packet level, why DNS has to go through the tunnel separately, how IP masking works (it is not magic, it is NAT), and how the three major protocols differ in their plumbing.

Written for people who want to understand the system without a networking degree.

The 30-second version

You install a VPN client and hit connect. Behind the scenes:

  1. Client and server do a handshake. They prove who they are to each other and agree on shared encryption keys. One or two round trips.
  2. Your OS routes traffic into the tunnel. The client tells the OS "send everything through this virtual network adapter." The tunnel is now the default route.
  3. Each packet gets encrypted, wrapped, and sent. The original packet (destined for, say, Wikipedia) gets encrypted and put inside a new outer packet addressed to the VPN server. From the network's perspective, you are only talking to the VPN server.
  4. The server unwraps and forwards. It decrypts the inner packet, sees Wikipedia as the destination, and sends it on with its own IP as the source. Wikipedia replies to the server, which reverses the process.
  5. DNS goes through the same tunnel. Looking up wikipedia.org runs over the encrypted tunnel and resolves at the VPN provider's DNS server, not your ISP's.

That is the whole pipeline. The rest is variation in how each step is implemented.

Step 1: the handshake and key exchange

Encryption only works if both sides agree on a key. The handshake establishes that key over a network where every byte is potentially being watched.

Modern VPN protocols use Diffie-Hellman or its elliptic-curve variant X25519 for this. Two parties each generate a public-private keypair, exchange the public halves, and independently compute the same shared secret without that secret ever crossing the wire. An eavesdropper who captures every byte cannot derive it.

Each protocol implements this differently:

WireGuard. One round trip. Your client sends a Noise-protocol initiation containing your ephemeral public key and your identity (encrypted with the server's static public key). The server replies with its ephemeral public key. Both sides derive symmetric session keys via HKDF and start sending data. Handshake time on a healthy network: 100ms or less.

OpenVPN. A full TLS 1.2 or TLS 1.3 handshake, the same back-and-forth a browser does when connecting to an HTTPS site. ClientHello, ServerHello, certificate exchange, key exchange, finished. More round trips, more bytes, more CPU. Handshake time: 500ms to over a second on slower networks.

VLESS Reality. A TLS 1.3 handshake to a real legitimate website (the SNI target). The Reality protocol cryptographically binds the connection to that target's TLS certificate while routing traffic to a different VPN server. To a network observer, the handshake is indistinguishable from someone visiting the SNI target. VPN authentication happens inside the TLS session via UUID identity.

After the handshake, each side has symmetric session keys. Symmetric encryption is fast; asymmetric (public-key) is slow. The handshake sets up the cheap symmetric session efficiently. HTTPS uses the same pattern for the same reason.

Step 2: encapsulation, not physical tunneling

The word "tunnel" makes people imagine some physical separation, a private road parallel to the public internet. The reality is more interesting. A VPN tunnel is just encapsulation: putting one packet inside another packet.

Picture a regular IP packet headed to Wikipedia. It has a header (source: your device, destination: Wikipedia's IP) and a payload (your HTTP request). When the tunnel is up, your client takes the whole packet, encrypts it, and stuffs it inside a new outer packet addressed to the VPN server. The original packet is now hidden inside the new envelope.

Routers between you and the VPN server only see the outer envelope. They route based on the outer header, which says "send this to the VPN server." Once it arrives, the server decrypts the payload, finds the original packet inside, and forwards it onward.

This trick (one IP packet wrapping another) is the mechanism behind every "tunneling" protocol on the internet. GRE, IPsec, WireGuard, OpenVPN: variations on the same theme. The encapsulation happens at the virtual network adapter the VPN client installs (Wintun on Windows, TUN on Linux, utun on macOS). Your OS sees a regular network interface; the VPN client intercepts every packet routed there, encrypts it, and sends the result through the real network.

Step 3: encryption in transit

Once the handshake produces session keys, the protocol uses them to encrypt every packet of traffic. The cipher choices matter for both security and performance.

ChaCha20-Poly1305. Stream cipher with built-in authentication. Fast on devices without hardware AES acceleration (older phones, low-end routers). WireGuard uses this exclusively. OpenVPN supports it. ChaCha20 is the modern default for protocols designed in the last decade.

AES-256-GCM. The block-cipher counterpart. AES-NI hardware instructions on every Intel and AMD CPU since 2010 make this the fastest option on desktop hardware. OpenVPN's modern default. WireGuard does not use it (the WireGuard authors deliberately chose ChaCha20 to avoid the AES-NI dependency for embedded use).

AES-128-GCM. Same family, smaller key. Marginally faster, marginally less paranoid. Modern protocols default to 256.

The "GCM" and "Poly1305" parts are important: they are authentication tags. Encryption alone tells you that an attacker cannot read your traffic; authentication tells you that an attacker cannot modify it without detection. A VPN that encrypts but does not authenticate is broken. Every modern protocol uses authenticated encryption (AEAD).

Each packet is encrypted independently with a counter that prevents replay attacks. If an attacker captures a packet and resends it later, the counter check fails and the receiver discards it.

Step 4: DNS resolution through the tunnel

DNS is the system that turns wikipedia.org into an IP address. By default, your OS sends DNS queries to whichever resolver was handed out by DHCP, which is usually your ISP's. That happens before any web traffic.

Without DNS handling, a VPN leaks badly. You connect the tunnel, your browser asks the OS to look up bank.com, the OS sends the lookup to your ISP's DNS server (outside the tunnel), and your ISP gets a list of every domain you visit despite the encrypted tunnel doing its job for the actual traffic.

A correctly implemented VPN client routes DNS through the tunnel. Several mechanisms:

Replace the system DNS resolver. The client sets the DNS server on the OS network configuration to the VPN provider's resolver, which is only reachable through the tunnel. Standard approach.

Block DNS leaks via firewall rules. On Windows, the VPN client uses Windows Filtering Platform (WFP) filters to block DNS queries on any interface other than the tunnel. Belt-and-suspenders: even if something tries to send a DNS query out the wrong interface, the firewall drops it.

Disable smart-multi-homed name resolution. Windows by default sends DNS queries to multiple network interfaces in parallel and uses whichever responds first. This Windows feature, called SMHNR, leaks DNS queries to the original ISP-provided resolver every time you make one, even with a VPN active. A correct VPN client disables SMHNR via NRPT (Name Resolution Policy Table) or registry entries.

Block IPv6 if the tunnel is IPv4-only. If your network supports IPv6 and the VPN does not tunnel IPv6, IPv6 DNS queries skip the tunnel entirely. Either tunnel IPv6 or block it outright.

The DNS handling is where many VPNs fail in subtle ways. A VPN that encrypts traffic but leaks DNS hands your ISP the same metadata they would have without the VPN. We have a deeper post on DNS leaks if you want the gritty version.

Step 5: IP masking via NAT

When the VPN server forwards the inner packet to Wikipedia, it has to make a choice: how does Wikipedia know where to send the reply?

The answer is Network Address Translation (NAT). The VPN server rewrites the source IP of the outgoing packet from your tunnel-side IP to its own public IP, and records the mapping in a NAT table. When Wikipedia replies, the server looks up the mapping, rewrites the destination back to your tunnel IP, encrypts the packet, and sends it back through the tunnel. Same NAT your home router uses, just at scale.

From Wikipedia's perspective, the source of your request is the VPN server's public IP. Wikipedia cannot derive your home IP from that connection. They can derive that you are using a VPN if they cross-reference the IP against known VPN exit lists (most large sites do for fraud detection), but they cannot identify you specifically.

This also explains why "shared IPs" are a privacy plus and a usability minus. If hundreds of people are exiting through the same IP, traffic to a given site cannot be attributed to any one of them. The downside: shared IPs are more likely to be flagged or rate-limited by sites that distrust them (some banks, some captchas, some streaming services).

Step 6: the OS routing trick

How does the VPN client convince your OS to send traffic into the tunnel? Not by intercepting individual sockets. The standard approach is to install a virtual network adapter and modify the routing table so that adapter becomes the default route.

The client adds:

  • A new default route through the virtual adapter (often as split routes 0.0.0.0/1 and 128.0.0.0/1 to override the existing default without removing it).
  • A specific route for the VPN server's IP through the original physical adapter, so the encrypted outer packets can actually leave your device.
  • DNS server set to the tunnel-side resolver.

Every outbound packet (except those addressed to the VPN server) now gets routed to the virtual adapter. The VPN client picks them up, encrypts, encapsulates, and sends the result via the physical adapter. The kill switch feature lives at this same layer: if the tunnel drops, the client either keeps the routes pointing at a non-existent gateway (traffic blackholes) or installs firewall rules that block traffic until the tunnel reconnects.

How protocols differ in plumbing

We covered the common skeleton. Each protocol fills in the details differently.

WireGuard. UDP only. Fixed-size handshake (148 bytes initiation, 92 bytes response). ChaCha20-Poly1305 only. Roams across networks gracefully because session state is keyed by the static identities, not the IP/port pair. Userspace implementations exist (boringtun, the one Fexyn uses) but kernel implementations are also available. Connect time is the lowest of the three: under a second, often under 500ms warm.

OpenVPN. UDP or TCP. TLS-based control channel, separate data channel. Configurable cipher (modern: AES-256-GCM or ChaCha20-Poly1305). Userspace only, larger codebase, more configuration knobs. Slower handshake, bigger per-packet overhead. The compatibility veteran: works on networks where WireGuard does not, especially in TCP mode where the connection looks like a generic encrypted TCP stream.

VLESS Reality. TCP only. The whole point is to look like ordinary HTTPS to a network observer. The handshake is a real TLS 1.3 handshake to a real legitimate site (configured as the SNI target on both client and server). Inside that TLS session, the VLESS protocol carries identity (a UUID) and traffic. The Vision flow (xtls-rprx-vision) we use adds a low-overhead inner framing for the VPN traffic. Slower than WireGuard for raw throughput but survives in environments where WireGuard and OpenVPN are blocked outright.

A modern client picks per-network. Fast, open networks: WireGuard. Networks that block WireGuard but allow generic TLS: VLESS Reality. Networks that block both but allow OpenVPN-shaped traffic: OpenVPN as the last fallback. Fexyn's rotation engine does this automatically.

What your ISP sees, with VPN on vs off

VPN off. Your ISP sees the IP of every server you connect to, the domains you look up via DNS, and how much data flows when. They cannot read HTTPS content but can read SNI (the domain in TLS handshakes), which gives them the same domain visibility from a different angle.

VPN on, well-implemented client. Your ISP sees one IP (the VPN server), encrypted bytes flowing to it, and timing patterns. DNS queries go through the tunnel, so the ISP cannot read them. SNI is no longer visible for your real traffic, because the only TLS handshake the ISP sees is to the VPN server (or to the SNI target with Reality, which is usually a popular site that reveals nothing about your destination). Your ISP can usually fingerprint that you are using a VPN, but not which sites you visit through it.

The trust shift is visible in this list. Your ISP loses everything they had. The VPN provider gains it. Logging policy and provider trust matter more than any single technical feature.

Performance overhead

A VPN adds two costs: encryption work on your CPU, and an extra hop on your traffic path.

CPU. Modern AES and ChaCha20 move at multiple gigabits per second on a single core. For typical home internet (under 1 Gbps), the CPU overhead is invisible.

Latency. Adding a hop adds latency proportional to the geographic distance between you, the VPN server, and the destination. A nearby exit adds 5-20ms; a transcontinental hop adds 100ms or more. Browsing and streaming absorb this fine; competitive gaming does not.

Throughput. WireGuard adds about 60 bytes per packet (outer IP/UDP headers plus authentication tag), roughly a 4% framing overhead on a 1500-byte MTU. OpenVPN's overhead is closer to 10%. The actual throughput hit is usually larger than the per-packet math because the VPN server can be a bottleneck if it is undersized for its user count. In our own testing, WireGuard costs 5-10% of total bandwidth on a healthy connection; OpenVPN costs 15-25%. VLESS Reality sits between them with more variance because of TLS framing.

The bottom line

A VPN works by encrypting and encapsulating your traffic, sending it through a chosen server, and forwarding it onward with NAT. The handshake establishes session keys. The encapsulation makes routers see only the outer packet. DNS, kill switch, and routing table changes catch the edge cases that would otherwise leak your real network identity.

The mechanism is well-understood and not magical. The privacy benefit depends on which party you are routing through, and what they do with what they see. Pick a protocol matched to your network conditions, a provider matched to your trust requirements, and a client that handles the leak vectors (DNS, IPv6, SMHNR, kill switch) properly.

FAQ

Encrypted, encapsulated IP packets. Your original packet (with its real source and destination) is encrypted and put inside a new outer packet addressed to the VPN server. To anyone watching the network, the outer packet is all they see.

The handshake step uses public-key cryptography (X25519 in WireGuard, RSA or ECDHE in TLS for OpenVPN and Reality) to establish symmetric session keys without those keys ever crossing the wire. Once both sides have the session keys, every packet is encrypted with a fast symmetric cipher (ChaCha20-Poly1305 or AES-256-GCM) that includes built-in authentication.

DNS lookups happen before web traffic and can leak independently of the tunnel. By default, your OS sends DNS to whichever resolver was handed out by DHCP, which is usually your ISP. A VPN client has to actively redirect DNS into the tunnel and block any leakage paths (Windows SMHNR, IPv6 fallback, browser DoH bypassing OS DNS).

TUN is a virtual layer-3 (IP) interface; TAP is a virtual layer-2 (Ethernet) interface. VPNs almost always use TUN because they only need to ferry IP packets. TAP is for use cases that need Ethernet emulation, like bridging local networks.

Yes, often. WireGuard packets have a recognisable shape; OpenVPN over UDP has a distinctive header. Even encrypted traffic has size and timing patterns that fingerprinting tools can use. VLESS Reality is the most resistant because it mimics real TLS to a real popular site. "Encrypted" and "undetectable" are different problems.

Without a kill switch, your OS falls back to the original default route and traffic flows in the clear from your real IP. With a kill switch, the client either keeps the routing table pointed at the (now non-existent) tunnel gateway (traffic blackholes) or installs firewall rules that block all traffic until the tunnel reconnects.

Yes, against the network operator. The Wi-Fi access point sees an encrypted tunnel to the VPN server. They cannot read your traffic, see which sites you visit, or inject content. HTTPS already protects much of this; a VPN adds defence in depth and protects metadata like SNI.

Smaller handshake (one round trip vs several), smaller per-packet overhead, simpler codebase that is easier to optimise, and a deliberate single-cipher design that avoids negotiation overhead. WireGuard was designed in 2015 with the lessons of older protocols in mind.

On the network: no. They see encrypted bytes to your VPN server, same as your ISP would. On the device: maybe. If your work device has corporate management software (MDM, EDR), that software can read traffic at the OS level before it enters the VPN. Network-layer privacy is real; device-layer privacy depends on the device.

How does a VPN work? A clear technical explanation | Fexyn VPN