The labs on this site need JavaScript to run. The same content is written out below.

Learn › Networking › Addresses and names

URL structure

After I added the search term "A&B" to the URL, the server received only "A"

A URL is divided into the scheme, host, port, path, query, and fragment at fixed symbols (://, :, /, ?, #, &, =). These symbols act as separators within a URL, so if you put the same character directly into a value, the URL gets cut at the wrong place.

Checked against real execution · All 300 combinations were run and checked against Node.js 24 URL, URLSearchParams, and the HTTP client (received by a test server on 127.0.0.1)

Concepts in this lab

URI and URL

A URI is a string that identifies a resource, and a URL is a URI that identifies the resource by its location (how to access it and its address). Most addresses used on the web are URLs, and every URL is a URI.

URL components

A URL is divided into several parts at fixed symbols. In https://shop.example:8443/shop/list?q=a#review, https is the scheme, shop.example is the host, 8443 is the port, /shop/list is the path, q=a is the query, and review is the fragment. If the port is not written, the connection uses the default port of the scheme (80 for http, 443 for https). The host and port decide where to connect, and the path and the query are carried in the request line to the server. The fragment is used only by the browser to find a position in the document it received, and it is not sent to the server.

Percent-encoding

A way of writing a character that has a special meaning in a URL, or that cannot be used in a URL, as % followed by two hexadecimal digits. The character is converted to UTF-8 bytes and each byte is written as %XX, so & becomes %26 and one Korean character becomes three %XX groups. If you put a character such as &, =, #, or + directly into a query value, it is read as a separator and the value is cut off or changed, so you must encode a value before you insert it. In a query, a space is sometimes written as +, which is why the + character itself must be written as %2B.

Query string and form data

The query string is the part of a URL after the ? in which name=value pairs are joined with &, and form data is text of the same shape sent in the request body. When form data is sent, the Content-Type header is set to application/x-www-form-urlencoded. When an HTML form is submitted with GET, the values are carried in the query string, and when it is submitted with POST, they are carried as form data in the body. The servlet container combines the two and provides them as "request parameters", so server code takes the values in the same way without telling which of the two they came from. A JSON body is not included in the request parameters.

Relative URL

A URL that leaves out the scheme and the host and gives only part of a URL. The browser fills in the missing parts from the URL of the current document, which is the base URL, to make a complete URL. If it starts with /, it replaces the whole path; otherwise it replaces the last segment of the base path, and .. means one level up. The result depends on whether the base path ends with /. The URL detail relative to /shop/list is /shop/detail, but relative to /shop/list/ it is /shop/list/detail.

Try these

• Default settings — The search term A&B is appended as it is. The server receives only A as the value of q.
• With encoding — & becomes %26, and the value arrives intact.
• A # in the search term — Everything after # becomes the fragment and is not sent to the server.
• A + in the search term — The server reads + as a space.
• Korean text and a space — See which characters the browser converts for you, and what they look like.
• A URL with a port — It connects to port 8443, which is not the default port, and the port is also added to the Host header.
• Writing the default port anyway — :443 is the same as leaving the port out, so it disappears from the normalized URL.

Running with the default settings

1. Building the URL: The search term ("A&B") was appended to the URL exactly as the characters are. The resulting text is https://shop.example/shop/list?q=A&B&sort=price#review.
2. Scheme: The part from the start up to ://, which is https. It decides which protocol is used to connect.
3. Host and port: The part after :// up to the first / is where to connect. The host is shop.example. No port is written, so it connects to port 443, the default port of https.
4. Path: The part after the host and port, which is /shop/list. It points to what is being requested inside the server, and it ends before the first ?.
5. Query: The part after ? up to the first #, which is q=A&B&sort=price. This is where the name=value pairs for the server are written, joined with &.
6. Fragment: Everything after the first #, which is review. It marks a position inside the document that was received, and only the browser uses it. It is not sent to the server.
7. What goes to the server: The browser connects to shop.example on port 443 and sends the request line (GET /shop/list?q=A&B&sort=price HTTP/1.1) and the Host header (Host: shop.example). The request line contains only the path and the query. The fragment is not carried anywhere.
8. Server query parsing: The query is split at &, each piece is split into a name and a value at the first =, and then + is turned back into a space and %XX into the original character. The result is q="A", B="", sort="price". The & inside the search term was read as the symbol that separates pairs. The value of q was cut off at "A", and an unrelated name, B, was created.
9. Resolving the relative URL: The link on this page (href="detail") gives only part of a URL, so the browser fills in the missing parts using the current URL as the base. It does not start with /, so the last segment of the current path (/shop/list) is replaced with detail. The query and the fragment are not inherited. The result is https://shop.example/shop/detail.

Result of this run

• &, =, #, +, %, and ? are characters that act as separators within a URL. If a value can contain such characters, you must always encode it before inserting it. In JavaScript, URLSearchParams or encodeURIComponent does this, and in Java, URLEncoder.encode does. Code that builds a URL by joining characters by hand usually works, and then fails on the day such a value arrives.
• The fragment (the part after #) does not go to the server. It is not in the server log, and server code cannot read it. The query, on the other hand, goes to the server and may be kept in its access log, so do not put values such as passwords in the query.

Code

// JavaScript. Paste it into the console of the browser developer tools to run it.
const keyword = 'A&B';

// Append the search term as the characters are
const url = new URL('https://shop.example/shop/list?q=' + keyword + '&sort=price#review');

console.log(url.href);                                   // the URL as the browser normalized it
console.log(url.protocol, url.hostname, url.port, url.pathname, url.search, url.hash);
console.log([...url.searchParams]);                      // the query read as names and values (the same rules as the server uses)
console.log(new URL('detail', url).href);                // the URL that the link (href="detail") points to

Assumptions of this model

All 300 combinations were run on Node.js 24 and checked. The result of splitting the URL, the normalized URL, and the resolution of relative URLs were checked against the values produced by the Node.js URL (an implementation of the WHATWG URL standard, which browsers follow). The connection port, the request line, and the Host header were checked by sending a request to that URL with the Node.js HTTP client, redirecting only the connection to a test server on 127.0.0.1, and receiving the characters that were actually sent (no DNS lookup for shop.example and no outside connection was made, and even the https URLs were received without encryption). The values the server received are the query of that request line parsed with URLSearchParams. Other servers such as Tomcat use the same rules (application/x-www-form-urlencoded), which split at & and = and turn + and %XX back, but this lab did not run them separately. Parts that differ from server to server, such as whether a malformed %XX is treated as an error, are not covered. encodeURIComponent turns a space into %20, not +, and in a query both are read as a space (this difference was also confirmed by execution). The rules for splitting a URL are in RFC 3986 section 3, removing the fragment before making the request is in section 3.5, resolving relative URLs is in section 5.2, and the request line and the Host header are in RFC 9112 section 3.2 and RFC 9110 section 7.2. That clicking a link that starts with # does not send a new request to the server was written down from the behavior of browsers (fragment navigation in the HTML standard) and was not run. URLs that contain a user name, IPv6 addresses, and domain names written in Korean are not covered.

Previous lab: Layers and encapsulation · Next lab: DNS