The labs on this site need JavaScript to run. The same content is written out below.

Learn › Spring › Other features

Calling external APIs and timeouts

When a call to another server gets no answer, how long does it wait?

The order server calls the API of the payment server. The calling thread stops and waits until the result comes back, and a timeout sets how long it waits before giving up. There are two timeouts. The connect timeout measures the time until the connection is established, and the read timeout is the limit on how long it waits, after the connection is established, while no data arrives. If you do not set them, the client never gives up on its own. Change the state of the remote server and the timeouts to see which timeout expires and when. The 2 seconds, 3 seconds, and 10 seconds on the screen are example values.

Partly checked against real execution · All 24 combinations were actually called with the RestClient of Spring 7.0.9 and checked against the results. The 4 combinations with a connection that gets no answer can differ depending on the network environment

Concepts in this lab

Connect timeout and read timeout: These are the two time limits that set how long a call to another server waits before giving up. The connect timeout is the limit on waiting until the TCP connection to the remote server is established, and the read timeout is the limit on waiting with no response data arriving after the connection is established. The two measure different intervals, so setting only the connect timeout does not protect against the case where the connection is made but the response is late. If no timeout is set, a slow remote server keeps the calling thread occupied, and when the thread pool runs out, the failure spreads to the calling side. A read timeout does not mean that the remote server did not handle the request, so you must watch out for duplicate handling when you send the same request again.
RestClient: This is the client that Spring provides for calling the HTTP API of another server. You build a request by chaining methods, as in client.get().uri(url).retrieve().body(type), and a message converter converts the response body to an object and returns it. It is synchronous, which means that the calling thread waits until the response arrives, and it is the successor to RestTemplate, which has been in use for a long time. A 4xx response is turned into an HttpClientErrorException, a 5xx response into an HttpServerErrorException, and a connection failure or a timeout into a ResourceAccessException, and that exception is thrown. The actual communication is done by a request factory, and timeouts are also set on the request factory.
Tomcat's threading model: Tomcat manages its request processing threads in a pool, and one thread takes one request and handles it until the response is finished. The number of requests it can handle at the same time is limited by the maximum number of threads (200 by default), and when all threads are in use, new requests wait. A thread stays occupied even while the request is waiting on a database or an external call.
Three-way handshake: The three-step process used to establish a TCP connection. The client sends a connection request (SYN), the server sends its acceptance together with its own connection request (SYN+ACK), and the client acknowledges it (ACK). After both sides have confirmed that they can send to and receive from each other, data transfer starts.
HTTP status codes: A three-digit number that shows the result of the server's handling of a request. 2xx means success, 3xx means a move to another location (redirection), 4xx means a problem with the client's request, and 5xx means an error on the server side. Common ones are 200 (success), 201 (created), 400 (bad request), 401 (authentication required), 403 (no permission), 404 (not found), and 500 (server error).

Try these

Default settings — Only the connect timeout is set. It does not cover a slow response, so the thread waits 10 seconds.
Setting the read timeout as well — If the response does not arrive within 3 seconds, the thread stops waiting.
Server that does not answer, no timeouts — The thread stays occupied until the operating system gives up.
Server that does not answer, read timeout only — The read timeout is of no use during the connection phase.
Server that does not answer, connect timeout — The call gives up after 2 seconds and releases the thread.
When the server program is not running — The refusal comes back right away, so the call does not wait for a timeout.
When a 404 or 500 comes back — The response arrived, but in the calling code it shows up as an exception.

Running with the default settings

Calling client.get() ... body(String.class): One thread of the order server starts calling the payment server. RestClient holds this thread until the result comes back, so during that time the thread cannot handle other requests.
Connection attempt: Before the HTTP request can be sent, a TCP connection to the payment server must be established. The connection request is sent. The 2-second connect timeout starts counting from this point.
Connection established: The payment server accepts the connection right away. The job of the connect timeout ends here. However long things take after this, the connect timeout does not expire.
GET /payments/1 sent: The request is sent over the established connection, and the thread waits for the response. No read timeout is set.
No read timeout: Nothing stops the wait. The job of the connect timeout (2 seconds) ended when the connection was established, so it is of no use here. The thread stays occupied for 10 seconds, until the response arrives.
Response arrives: After 10 seconds, the payment server sends a 200 response. RestClient converts the body to a string and returns it. The result is a success, but one thread could do no other work for 10 seconds.

Result of this run

If you do not set a read timeout, the wait has no limit. When the payment server slows down, the threads of the order server stay occupied for that long, and when many requests come in at once, all the threads in the thread pool become occupied and the order server itself can no longer respond. The connect timeout alone does not protect against a slow response. The two timeouts measure different intervals, so you must set both.

Code

SimpleClientHttpRequestFactory factory = new SimpleClientHttpRequestFactory();
factory.setConnectTimeout(Duration.ofSeconds(2));   // connect timeout
// no read timeout is set (waits with no limit)

RestClient client = RestClient.builder()
        .requestFactory(factory)
        .build();

try {
    String body = client.get()
            .uri("http://pay-server/payments/1")
            .retrieve()
            .body(String.class);            // this thread waits here until the result comes back
} catch (HttpClientErrorException e) {      // 4xx response
} catch (HttpServerErrorException e) {      // 5xx response
} catch (ResourceAccessException e) {       // connection failure, timeout
}

Assumptions of this model

All 24 combinations were actually called with the RestClient of Spring Framework 7.0.9 (the request factory is SimpleClientHttpRequestFactory, on JDK 21), and it was confirmed that the result, the kind of exception, the cause of the exception, the time waited, and whether the remote server handled the request match the screen. The remote server is a Spring Boot 4.1.1 server (embedded Tomcat) started on 127.0.0.1 of the same PC, with a URL that responds late on purpose. The times on the screen are examples. The real check was run with the times reduced to one hundredth (connect timeout 200ms, read timeout 300ms, slow response 1000ms), and the time waited was checked to come out in that proportion. "Connection refused" was checked with a port on 127.0.0.1 that no program is listening on. "No answer to the connection request" cannot be produced with 127.0.0.1, so it was checked with 192.0.2.1 (RFC 5737), an address that is reserved for documentation and is not assigned to any machine. These 4 combinations can give different results depending on the network the PC is on (if the network is disconnected, the call fails right away). The roughly 21 seconds that the operating system took to give up when there was no connect timeout is the value seen on Windows 11, and it differs depending on the operating system and its settings. That the wait has no limit when no timeout is set follows the java.net.URLConnection documentation (a timeout of 0 means infinite), and execution confirmed only that the call waits until the response arrives. The read timeout is not a limit on the time the whole response takes but a limit on the time spent waiting with no data arriving, so it may not expire for a response that keeps arriving a little at a time (this follows the same documentation and was not verified by execution). How timeouts are set and what their defaults are differ depending on the kind of request factory. The check was done one call at a time, so the situation where requests pile up and the thread pool runs out was not verified by execution. The model is simplified to the case where a new connection is established for every call, and it does not cover a connection pool that reuses connections and its wait time, retries, or circuit breakers.

Previous lab: Testing · Next lab: Cache abstraction

Concepts in this lab

What this lab shows

Settings

Predict the result before running

Code

Diagram

Run comparison