The labs on this site need JavaScript to run. The same content is written out below.

Learn › Java › Lambdas and streams

Parallel streams and the common pool

Someone else's parallel stream slows down my code

parallelStream() does not create a thread pool of its own. It runs in the single common pool that the whole JVM shares. If some code runs tasks that wait for a long time in it, all the threads of the common pool are occupied, and other parallel streams that have nothing to do with that code slow down too.

Checked against real execution · Values actually measured on JDK 21

Concepts in this lab

ForkJoinPool and the common pool

This is a thread pool used to split a large task into small tasks, process them on several threads, and combine the results. The JVM has a common pool (commonPool) that everything shares, and parallel streams run in it by default. If you run a long task there, other tasks that use the same pool also slow down.

Stream API

This API expresses filtering, transformation, and aggregation over data such as collections in a declarative way. You chain intermediate operations such as filter and map, and finish with a terminal operation such as collect or forEach. Intermediate operations do not run until the terminal operation is called (lazy evaluation), and a stream does not modify its source data and can be used only once.

Thread pool

This structure keeps a fixed number of threads and reuses them, so that threads are not created and destroyed every time they are needed. It reduces the cost of creating threads, and by limiting the number of threads that run at the same time, it prevents excessive memory use and context switching. Tasks to be processed pile up in a queue, and an idle thread takes one and runs it. Tomcat's request handling and a DB connection pool follow the same idea.

Blocking and non-blocking

Blocking means that the calling thread is stopped until the called task finishes and cannot do other work. Non-blocking means that the call returns right away even though the task has not finished, so the thread can do other work. A typical JDBC call is a blocking call: the thread is stopped until the DB response arrives.

Try these

• Default settings — A occupies the common pool. B becomes 4 times slower.
• When only B runs — B has the whole common pool to itself, so it is fast.
• Moving A to a separate pool — B becomes fast again.

Running with the default settings

1. The common pool (ForkJoinPool.commonPool) has 3 worker threads. Its default size is "number of CPU cores - 1", and the thread that calls a parallel stream also takes part in the work itself. Every parallel stream in the JVM shares this one pool.
2. Request A calls an external API inside a parallel stream. All 3 worker threads of the common pool pick up one task of A each and stay stopped for 2 seconds at a time while they wait for the response. They are not computing anything. They are only waiting, and yet the threads are occupied.
3. Request B runs 8 tasks of 0.1 seconds each with parallelStream(). It splits the work and hands it to the common pool.
4. All the worker threads of the common pool are occupied by A, so there is no thread to take the tasks of B. The thread that called B runs the 8 tasks one by one, alone. It is a parallel stream, but in effect it runs sequentially.
5. B has finished. Time taken: 862ms. That is about 4 times as long as when B runs alone (217ms). Not a single line of the code of B has changed.

Result of this run

• The common pool is shared by the whole JVM. If one place puts tasks that wait for a long time into a parallel stream, the parallel streams of other code that knows nothing about it slow down together with it, and so does CompletableFuture (when no pool is specified). The code that causes the problem and the code that slows down are different, so the cause is also hard to find.
• A parallel stream pays off only when a computation that uses the CPU is applied to a large amount of data. That use is also why the size of the common pool is matched to the number of cores. When there are few elements, when the tasks are light, or when waiting is mixed in, a parallel stream can be slower than a sequential stream. In the request-handling code of a web server, there is almost never a reason to use one.

Code

// Request A: calls an external API inside a parallel stream (2 seconds per call)
orders.parallelStream()
      .forEach(o -> deliveryApi.track(o));      // a thread of the common pool is occupied for 2 seconds each time

// Request B: 8 tasks of 0.1 seconds each
items.parallelStream().forEach(this::resize);

Assumptions of this model

The three times that B took (217ms, 862ms, 218ms) are values measured by actually running the code on JDK 21 on this PC, with the number of worker threads in the common pool set to 3. The 0.1-second tasks and the 2-second calls were replaced with Thread.sleep. That the default size of the common pool is "number of cores - 1" was checked on JDK 21 on this PC (with 12 processors, the parallelism of the common pool is 11). That parallel streams use the common pool is shown by the measurement above (B slowed down when someone else's parallel stream occupied the common pool).

Previous lab: Collecting a stream · Next lab: Processes and threads