The labs on this site need JavaScript to run. The same content is written out below.

Learn › Java › Threads and concurrency

Blocking, non-blocking, and virtual threads

What a thread does while it waits

Every request spends some time waiting for a response from a database or an external API. Compare three approaches: holding on to the thread during that time (blocking), releasing the thread and continuing when the response arrives (non-blocking), and virtual threads, where you write the code in the blocking style but the operating system thread is released while the code waits.

Checked against real execution · Checked against values actually measured on JDK 21

Concepts in this lab

Blocking and non-blocking

Blocking means that the calling thread is stopped until the called task finishes and cannot do other work. Non-blocking means that the call returns right away even though the task has not finished, so the thread can do other work. A typical JDBC call is a blocking call: the thread is stopped until the DB response arrives.

Synchronous and asynchronous

Synchronous means that the side that requests a task waits until it receives the result and only then moves on. Asynchronous means that the requester starts the task and moves on to the next work without waiting for the result, and the result is delivered later through a callback, a Future, or a similar means.

Virtual thread

This is a lightweight thread that became a standard feature in JDK 21. An existing thread is tied one-to-one to an operating system thread, so you cannot create many of them, but this kind is managed by the JVM and takes turns running on a small number of operating system threads. While it waits in most blocking calls, it gives up the operating system thread, so even the existing simple coding style can handle many concurrent requests. However, in JDK 21 through 23, blocking inside synchronized held on to the operating system thread (pinning), and this was resolved in JDK 24.

Spring WebFlux and the reactive stack

This is Spring's non-blocking web framework for handling many concurrent requests with a small number of threads. Unlike Spring MVC, which uses one thread per request, it does not hold a thread while waiting for I/O and continues processing when the data is ready. It pays off only when the whole call path is non-blocking, so it does not fit blocking technologies such as JPA.

Event loop

This is a structure in which a small number of threads detect the I/O events that occur on many connections and process them in turn. There is no thread per connection, so one thread takes care of many connections. If you run a long task or a blocking call on such a thread, processing stops for every connection that the thread is responsible for.

Tomcat's threading model

Tomcat manages its request processing threads in a pool, and one thread takes one request and handles it until the response is finished. The number of requests it can handle at the same time is limited by the maximum number of threads (200 by default), and when all threads are in use, new requests wait. A thread stays occupied even while the request is waiting on a database or an external call.

Try these

• Default settings — 4 threads and 12 requests. The threads are tied up waiting, so the later requests wait in line.
• With an event loop — 2 threads keep all 12 requests in progress at the same time.
• Blocking calls on the event loop — The 2 threads are tied up, so only 2 requests make progress at a time. This is the worst combination.
• Virtual threads — The code stays in the blocking style, yet the requests are in progress at the same time.
• A larger thread pool — 10 threads and 30 requests. Even with more threads, requests wait in line just the same when their number grows as much.

Running with the default settings

1. 0.0s: Requests arrive (12 in total). 4 threads take one request each. The rest (8) wait until a thread is free.
2. 0.1s: Each thread waits for an external response. While it waits, the thread does no work, but it cannot take another request either (blocking).
3. 2.0s: The responses arrive. Requests finished: 4. The freed threads take requests that were waiting (still waiting: 4).
4. 4.0s: The responses arrive. Requests finished: 4. The freed threads take requests that were waiting (still waiting: 0).
5. 6.0s: The responses arrive. Requests finished: 4.

Result of this run

• In the blocking style, the number of requests that can be handled at the same time cannot exceed the number of threads. The longer the wait, the longer the threads are tied up. The code is simple and easy to debug, which is why this style is widely used.

Code

// Spring MVC: one thread handles each request from start to finish
// server.tomcat.threads.max=4
@GetMapping("/orders/{id}")
OrderDto get(@PathVariable long id) {
    Order order = orderRepository.findById(id);   // the thread stops and waits until the response arrives
    Delivery d = deliveryClient.track(order);     // (2s)
    return OrderDto.of(order, d);
}

Assumptions of this model

This model ignores the CPU time spent in handling a request and assumes that a request consists only of waiting for an external response. The number of OS threads that run the event loop and that carry the virtual threads is fixed at 2 (the real default follows the number of CPU cores). Virtual threads were finalized in JDK 21. In JDK 21 to 23 there was a limitation that a virtual thread waiting inside a synchronized block could not release its OS thread, and this was resolved in JDK 24. The times were actually measured on JDK 21 on this PC, with the wait shortened to 0.2s. With 12 requests, a pool of 4 threads took 619ms (handled in three rounds), blocking on 2 threads took 1,227ms, virtual threads took 216ms on 2 OS threads, and the approach that only registers the wait took 201ms on 2 threads. These are the same ratios as in the model (6s, 12s, 2s, 2s). The measurement did not run Spring WebFlux itself. It reproduced the same principle with the JDK alone.

Previous lab: Sequential and parallel calls · Next lab: Stack and heap