The labs on this site need JavaScript to run. The same content is written out below.

Learn › Security › Web attacks and defenses

Rate limiting

Three meanings of "accept only 5 requests per second"

When requests arrive all at once, the server rejects some of them to protect itself (429 Too Many Requests). Even the same limit of "5 requests per second" lets different requests through, depending on how you count.

Conceptual model · A model built to the widely used definitions of the algorithms

Concepts in this lab

Rate limiting

Limiting the number of requests that are allowed in a given period of time. It stops a specific user or client from sending excessive requests and affecting the whole system, and it makes the system accept requests only within the range it can handle. Requests over the limit usually get a response with the 429 status code.

HTTP status codes

A three-digit number that shows the result of the server's handling of a request. 2xx means success, 3xx means a move to another location (redirection), 4xx means a problem with the client's request, and 5xx means an error on the server side. Common ones are 200 (success), 201 (created), 400 (bad request), 401 (authentication required), 403 (no permission), 404 (not found), and 500 (server error).

API gateway

The single entry point that receives all client requests first and forwards them to the right internal service. It handles common functions such as authentication, rate limiting, and logging in one place. Clients do not need to know the layout and addresses of the internal services.

Try these

• Default settings — A fixed window. All 10 requests pass within 0.4 seconds, on both sides of the boundary.
• The same requests with a sliding window — No 1-second span lets through more than 5 requests.
• A sudden burst with a token bucket — As many requests as there are saved tokens pass at once.
• Evenly, 8 per second — 3 requests over the limit are rejected every second.
• Even requests with a token bucket — Requests keep passing at the speed at which tokens are refilled.

Running with the default settings

1. Request #1 at 0.80s: The counter for the window starting at 0s is 1. That is 5 or less, so it passes.
2. Request #2 at 0.84s: The counter for the window starting at 0s is 2. That is 5 or less, so it passes.
3. Request #3 at 0.88s: The counter for the window starting at 0s is 3. That is 5 or less, so it passes.
4. Request #4 at 0.92s: The counter for the window starting at 0s is 4. That is 5 or less, so it passes.
5. Request #5 at 0.96s: The counter for the window starting at 0s is 5. That is 5 or less, so it passes.
6. Request #6 at 1.02s: The counter for the window starting at 1s is 1. That is 5 or less, so it passes.
7. Request #7 at 1.06s: The counter for the window starting at 1s is 2. That is 5 or less, so it passes.
8. Request #8 at 1.10s: The counter for the window starting at 1s is 3. That is 5 or less, so it passes.
9. Request #9 at 1.14s: The counter for the window starting at 1s is 4. That is 5 or less, so it passes.
10. Request #10 at 1.18s: The counter for the window starting at 1s is 5. That is 5 or less, so it passes.
11. Request #11 at 2.20s: The counter for the window starting at 2s is 1. That is 5 or less, so it passes.
12. Request #12 at 2.40s: The counter for the window starting at 2s is 2. That is 5 or less, so it passes.
13. Request #13 at 2.60s: The counter for the window starting at 2s is 3. That is 5 or less, so it passes.
14. Request #14 at 2.80s: The counter for the window starting at 2s is 4. That is 5 or less, so it passes.

Result of this run

• A fixed window resets its counter to 0 at every whole second, so if requests pile up just before and just after a boundary, up to twice the limit passes in a short time. It is the simplest to implement and uses little memory, but it has this gap.
• When you reject a request, it is good to send the 429 status code together with when the client may try again (the Retry-After header). If there are several servers, keep the counter in a shared store such as Redis so that the limit does not grow with the number of servers.

Code

// Fixed window: key = user + the current second
String key = "rate:" + userId + ":" + (now / 1000);
long count = redis.incr(key);            // add 1 to the request count of this second
if (count == 1) redis.expire(key, 1);    // disappears after 1 second
if (count > 5) return tooManyRequests(); // 429

Assumptions of this model

The limit is fixed at 5 requests per second. The behavior of the three algorithms follows widely used definitions. The sliding window is implemented by recording the time of every request (a sliding window log), and the token bucket assumes that tokens are refilled continuously as time passes. A real service counts separately for each user, IP address, or API key.

Previous lab: Information leakage · Next lab: Dependencies and the supply chain