Percent-encoding: how URLs handle characters they weren't built for

You've seen %20 in a URL. Probably %2F too, or that classic %3F when someone pastes a query string wrong. That % followed by two hex digits is percent-encoding — the web's way of stuffing arbitrary bytes through a channel originally designed for a narrow slice of ASCII.

Percent-encoding was documented in RFC 1630 in June 1994, before RFC 1738 specified URLs that December¹². It remains part of URL syntax today.

How the %XX triplet works

Take a byte. Write it as % plus two hexadecimal digits. Space (0x20) becomes %20. Hash (0x23) becomes %23. That's the entire idea.

The hex digits can be uppercase or lowercase — %2f and %2F both mean forward slash. RFC 3986 recommends producers use uppercase, but consumers must accept either³.

Strict RFC 3986 component-data encoding

Here's the part that catches people: percent-encoding operates on bytes, not characters. For ASCII, there's a clean 1:1 mapping — @ is byte 0x40, so its percent-encoded form is %40. Anything outside ASCII has to be converted to bytes first (usually using UTF-8), and then each byte gets its own %XX triplet.

The accented letter é (U+00E9) in UTF-8 produces two bytes: 0xC3 and 0xA9. So it becomes %C3%A9. A Chinese character like U+4E2D turns into three UTF-8 bytes and becomes %E4%B8%AD.

Unicode to percent-encoded bytes via UTF-8

Character classes in RFC 3986

The current URI standard is RFC 3986, published in January 2005 by Berners-Lee, Fielding, and Masinter³. It splits characters into categories that determine what needs encoding and where.

Unreserved characters never need encoding:

A-Z a-z 0-9 - . _ ~

Letters, digits, hyphen, period, underscore, tilde. If you percent-encode one of these, a conforming implementation should treat %41 identically to A³.

Reserved characters carry syntactic meaning:

• Gen-delims: : / ? # [ ] @
• Sub-delims: ! $ & ' ( ) * + , ; =

Whether a reserved character needs encoding depends on context. A / separates path segments, but RFC 3986 also permits it unencoded in a query. In a form-encoded query, & separates parameters; an ampersand inside a value needs %26 instead³⁴. This context-dependence is the trickiest part of the system.

Everything else — control characters, spaces, non-ASCII bytes, { } | \ ^ — always needs encoding.

Standards evolution (briefly)

URI encoding standards from 1994 to present

RFC 1738 (1994) documented %HH escaping with categories of "safe" and "unsafe" characters². It listed ~ as unsafe, affecting Unix home directory URLs. RFC 2396 (1998) moved ~ to unreserved and defined generic URI syntax⁵. RFC 3986 (2005) split reserved characters into gen-delims and sub-delims, formalized normalization rules, and trimmed the unreserved set to 66 characters³.

A companion spec, RFC 3987, defined Internationalized Resource Identifiers (IRIs) — identifiers that can contain Unicode⁶. Its IRI-to-URI mapping encodes non-ASCII characters as UTF-8, then percent-encodes the bytes. Browser URL parsing follows the WHATWG URL standard, which retains some legacy query-encoding behavior⁷.

Browsers may display Unicode characters while serializing URL components with percent escapes. Internationalized domain names use a different encoding scheme called Punycode, which is part of the broader IDN infrastructure.

The form encoding oddity

HTML forms do their own thing. When a form submits with method="GET" (or POST with the default enctype), the browser uses application/x-www-form-urlencoded⁴. It looks like percent-encoding but has one well-known deviation: spaces become + signs.

name=John+Doe&city=New+York

This dates back to RFC 1866 — the HTML 2.0 specification from 1995⁸. RFC 3986 knows nothing about + meaning space. In a URI path, + is a literal plus sign. But in form-encoded query strings, + and %20 both represent a space.

Form encoding percent-encodes bytes outside A-Z a-z 0-9 * - . _, with the space-to-+ exception⁴. It encodes both ! and ~; RFC 3986 classifies ! as reserved and ~ as unreserved.

I find this split annoying. The rules differ depending on whether you're building a path, a query parameter, or form data.

encodeURI vs encodeURIComponent

encodeURI vs encodeURIComponent comparison

encodeURI() encodes a complete URI. It preserves : / ? # @ ! $ & ' ( ) * + , ; =, along with letters, digits and - _ . ~. It does encode square brackets: [] becomes %5B%5D⁹.

encodeURIComponent() encodes a single component (a query parameter value, a path segment). It encodes the structural characters too, because inside a component, a / or & is data, not syntax¹⁰.

const url = "https://example.com/search?q=coffee & tea";

encodeURI(url);
// "https://example.com/search?q=coffee%20&%20tea"
// Bad: & is preserved, breaking the query string

"https://example.com/search?q=" + encodeURIComponent("coffee & tea");
// "https://example.com/search?q=coffee%20%26%20tea"
// Correct: & is encoded as %26

Rule of thumb: encodeURIComponent() for values, encodeURI() for whole URLs where the structure is already valid. Both leave ! ' ( ) * unescaped, so neither is a strict RFC 3986 component encoder⁹¹⁰.

Neither function produces form encoding — both use %20 for spaces, never +. For that, use URLSearchParams:

const params = new URLSearchParams({ q: "coffee & tea" });
params.toString(); // "q=coffee+%26+tea"

Python has a similar split: urllib.parse.quote() uses %20 for spaces and keeps / safe by default (path encoding), while urllib.parse.quote_plus() uses + for spaces and encodes slashes (form encoding)¹¹.

from urllib.parse import quote, quote_plus

quote("hello world")       # 'hello%20world'
quote_plus("hello world")  # 'hello+world'

Common pitfalls

Double encoding. If you encode a URL that's already percent-encoded, the % signs themselves get encoded: %20 becomes %2520. This happens when one layer of your stack encodes a value and another layer re-encodes it. RFC 3986 warns against this explicitly³. ORMs and HTTP client libraries are frequent offenders.

The plus sign trap. In a form-encoded query string, + means space. In a URI path, + is a literal plus. Decode a path segment with a form decoder and every + silently becomes a space. I've seen this bug in production more than once.

Wrong encoding context. Using encodeURI() on a query parameter value preserves &, which a form query parser treats as a separator. Using encodeURIComponent() on an entire URL encodes the :// and every / in the path, so it no longer represents that URL's structure. The function choice depends on what you're encoding and where it goes.

Non-UTF-8 legacy. UTF-8 is the default in modern URL APIs, but legacy encodings still matter. The character é is 0xE9 in Latin-1 (one byte, %E9) but 0xC3 0xA9 in UTF-8 (two bytes, %C3%A9). Mixing these up corrupts text. The WHATWG URL parser accepts an encoding parameter for legacy HTML query handling, so UTF-8 is not universal even in browsers⁷.

Updated: September 7, 2026