The labs on this site need JavaScript to run. The same content is written out below.

Fundamentals › Architecture and operations

Calling another server

How is a server calling another server different from calling a method?

A large service sometimes splits its features among several servers. If the order server that takes orders and the payment server that handles payments are different programs that run separately, the order server has to send a request (POST /pay) to the payment server over the network to make a payment. When you call a method in the same program, the one you call is always in the same program and the call finishes right away. But when you call a server across a network, the other side may be off and refuse the connection, or the other side may be slow and you may have to wait. So the code of the order server has to decide how long to wait for an answer (a timeout) and what to answer the client with when the call fails. In this lab, you change the state of the payment server and whether the waiting time is set, and see the response the client receives, the time it took, and what happened to the order server. The code box is not a program but a list of who sends what to whom, and the real run started an order server and a payment server with these settings and exchanged requests between them.

Checked against real execution · Started an order server and a payment server on Node.js 24.18.0, ran six combinations, and checked them

Concepts in this lab

Remote call

A remote call (a call between servers) is when the program of one server sends a request to another server over the network and uses a feature of that server. Section 3.3 of RFC 9110 calls a program that makes a connection and sends a request a client, and a program that accepts a connection and responds a server, and says that the same program can be a client on one connection and a server on another. So an order server is a server to a browser and a client to a payment server. Unlike a method call inside the same program, the other side may be off and the connection may fail, if the other side is slow you have to wait until the answer comes, and the answer may never come. So the code that makes the call has to decide how long to wait and what to do when it fails.

Connect timeout and read timeout

These are the two time limits that set how long a call to another server waits before giving up. The connect timeout is the limit on waiting until the TCP connection to the remote server is established, and the read timeout is the limit on waiting with no response data arriving after the connection is established. The two measure different intervals, so setting only the connect timeout does not protect against the case where the connection is made but the response is late. If no timeout is set, a slow remote server keeps the calling thread occupied, and when the thread pool runs out, the failure spreads to the calling side. A read timeout does not mean that the remote server did not handle the request, so you must watch out for duplicate handling when you send the same request again.

502 Bad Gateway

502 is a status code used when a server acting as a gateway or proxy receives an invalid response from the inbound server it accessed to fulfill the request (RFC 9110 section 15.6.3). The one that makes and sends this response is the proxy server the client connected to, not the origin server behind it. So when a client receives 502, it means the client reached the proxy server it connected to, and the problem is between the proxy server and the server behind it.

504 Gateway Timeout

504 is a status code used when a server, while acting as a gateway or proxy, does not get a timely answer from the inbound server it accessed to fulfill the request (RFC 9110 section 15.6.5). A server that calls another server can answer the client with this code when it gives up after passing its waiting limit. It is told apart from 502, which is used when an invalid answer is received from the inbound server. Receiving a 504 only means that the waiting has ended, not that the inbound server did not process the request. The inbound server may have finished processing after the limit was passed.

Try these

• Default settings — The payment server is on and replies after 0.1s. The order server receives the reply and answers the client with 200.
• Payment server off — The connection is refused, and the order server answers 502 right away.
• Payment server replies after 3s — When 1s is exceeded, the order server stops waiting and answers 504.
• Removing the waiting time — There is no limit, so it waits the full 3s, and the client also receives the response after 3s.
• Removing the waiting time, with the server off — There is nothing to wait for, so the result is the same as with the limit.

Running with the default settings

1. The client sends a POST /orders request to the order server. The service in this lab is split into the order server that takes orders and the payment server that handles payments, and the two are different programs that run separately. The # lines in the code box are not a program but a list of who sends what to whom.
2. While handling the order, the order server has to make a payment. The feature that makes payments is not inside the order server but in the payment server. If a feature is in the same program, you can just call a method, but a feature of another server can only be called by sending a POST /pay request to that server over the network. The order server is set to wait for the answer to this request for up to 1s only.
3. The payment server receives the POST /pay request and processes the payment. This payment server takes 0.1s to process. Meanwhile, the order server waits for the answer.
4. The payment server answers "Payment complete" with a 200 response. The order server receives this answer and knows the payment is done. The answer came before the 1s that was set, so the waiting did not hit the limit.
5. The order server returns "Order complete" to the client with a 200 response.
6. The client received the 200 response in about 0.1s. The result looks the same as calling one method, but in between there was a step of exchanging text with another server. For this step to go well, the payment server had to be on and had to answer in time.

Result of this run

• A server calling another server means sending a request over the network and waiting for the answer. The order server sent POST /pay to the payment server, received a 200 answer after 0.1s, and answered the client with 200 "Order complete".
• What differs from a method call is that the other side is a server that runs separately. This time the payment server was on and answered quickly, so the 1s limit was not used. The cases where it is off or answers late are in the next questions.

Code

# Client → order server (8080): POST /orders
# Order server → payment server (9000): POST /pay  (waits up to 1s)
# Payment server: replies "Payment complete" after 0.1s
# Order server: answers 502 for a connection failure and 504 for a timeout

Assumptions of this model

The code box is not a program that was run. It is text that lists what the client, the order server, and the payment server exchanged. The real run started the order server and the payment server separately on 127.0.0.1 with the http module of Node.js 24.18.0 (the ports were free ports chosen by the operating system, and 8080 and 9000 on the screen are example numbers), and made the client send POST /orders to the order server. The order server sends POST /pay to the payment server (it does not reuse connections), and we wrote it to answer 200 "Order complete" when the payment server answers, 502 on a connection failure (ECONNREFUSED), and 504 on a timeout. The payment server answers 200 "Payment complete" after 0.1s or after 3s, and a payment server that is off was imitated by starting a server and closing it to make a port that nobody listens to. Waiting for up to 1s was implemented by giving AbortSignal.timeout(1000) as the signal of http.request, and not setting it was implemented by not giving a signal. We ran all six combinations and checked against the model the response the client received and the time it took, the kind of error the order server met, whether the payment server received the request and when it finished processing the payment, and whether the connection was already cut off at that time; for each combination we also checked that the code on the screen is the same text as the text made from the request line the servers actually received, the waiting limit set on the order server, the delay setting of the payment server, and the status code for each error. The time it took is written on the screen as a rough value, and the values measured on the PC used to write this lab were about 0.1s (0.117s), immediately (0.003s or less), about 1s (1.01s), and about 3s (3.02s). When the payment server was off, the same result came out whether or not the waiting time was set. When a payment server that replies after 3s was given a 1s limit, this payment server finished processing the payment after 3s even after the order server stopped waiting. This payment server was written for this lab to finish processing without checking whether the connection was cut off, which does not mean that every server behaves like that. When the waiting time was not set and the payment server received the request but never answered, the order server did not give up even after 4s and the client did not receive a response either, but we did not look at what happens if you wait any longer. These are behaviors observed in this version of Node.js. What we checked against the documentation is section 3.3 of RFC 9110 (the definitions of client and server, and that the same program can be both), section 15.6.3 (502 is for when a server acting as a gateway or proxy receives an invalid response from an inbound server), and section 15.6.5 (504 is for when a server acting as a gateway or proxy does not receive a timely response from an inbound server), and http.request in the Node.js documentation (the signal option aborts a request in progress, while the timeout option does not abort the request and only adds a timeout event). Strictly speaking, the order server is not a proxy but a server doing its own work, so using 502 for a connection failure and 504 for a timeout is a choice of this code that follows the use defined by the specification in a broad way, and some servers may answer with another status code (for example, 503). That a method call in the same program has no step called connecting is an explanation that we did not measure by running. The screen covered only the simplest case of calling once, and it did not cover sending a failed call again, skipping a server that keeps failing instead of calling it, or undoing work that spans several servers. The difference between a connect timeout and a read timeout is covered in the in-depth lab "Calling external APIs and timeouts", a device that protects against a failing server in "Circuit breaker", how to undo the work of several servers in "Saga and outbox", and how to receive requests in one place and split them in "Gateway and gradual migration".

Previous lab: Processes · Next lab: Load balancers