URL Encoding Explained: Percent Encoding & RFC 3986

URLs are the navigation anchors of the web, but they are limited to a specific subset of ASCII characters. When your search queries contain spaces, symbols, or non-English characters, they must be converted using URL encoding. Let's break down how this percent-encoding process works.

1. The ASCII Limits of URL Syntax

URLs are designed to travel through diverse web protocols (like DNS, HTTP headers, and server routers). To prevent compatibility issues, the URI specification (RFC 3986) restricts URL characters to a subset of ASCII.

Any characters outside this set—such as emojis, Cyrillic scripts, or accented letters—must be encoded to ensure safe transport.

2. Reserved vs Unreserved Characters

RFC 3986 divides ASCII characters into two groups:

• Unreserved: Safe characters that never require encoding: alphanumeric characters (A-Z, a-z, 0-9) and symbols -, ., _, and ~.
• Reserved: Characters with syntactic meaning (like ? for query strings, & for parameters, and / for paths). These must be encoded if they are part of the data payload.

3. How Percent-Encoding Works

Percent-encoding represents unsafe characters with a % followed by the two-digit hexadecimal representation of their UTF-8 byte value. For example:

• A space (ASCII value 32) becomes %20.
• An ampersand (&) becomes %26.
• The forward slash (/) becomes %2F.

4. The Plus (+) vs %20 Space Dilemma

You may notice spaces represented as + in some URLs and %20 in others:

• %20 is the standard percent-encoding for spaces in RFC 3986.
• + is used inside form payloads (using the application/x-www-form-urlencoded content type) to make long query strings more readable.

5. How to Handle URL Encoding in Code

Most programming languages offer built-in encoding functions. In JavaScript, use encodeURIComponent() to encode parameter values safely, and decodeURIComponent() to recover the original string.