The labs on this site need JavaScript to run. The same content is written out below.

Learn › Networking › Transport layer

Flow control and congestion control

How does TCP decide how much to send at a time?

Even when TCP has a lot of data to send, it does not pour it all out at once. It sends only up to the smaller of two amounts: the amount the receiver can handle (the receive window) and the amount TCP estimates the network can handle (the congestion window). Respecting the first is flow control, and adjusting the second is congestion control.

Conceptual model · A model of the concept that follows the rules of RFC 5681. It was not checked against a real TCP run; it was checked in 153 settings against a separate calculation of the same rules, one ACK at a time

Concepts in this lab

Flow control and congestion control

Both are TCP functions that adjust the sending rate. Flow control adjusts the amount sent to match what the receiver can handle, so that the receiver does not overflow. Congestion control senses how congested the network is and decreases or increases the amount sent, to prevent congestion across the whole network.

Receive window and congestion window

These are the two limits that decide how much TCP can send at one time. The receive window (rwnd) is the value the receiver reports in its acknowledgments, saying "I can receive this much more right now," and it keeps the buffer of the receiver from overflowing. The congestion window (cwnd) is the value the sender manages on its own, based on its estimate of "the network can handle this much," and it is not reported to the other side. The sender sends so that the data not yet acknowledged does not exceed the smaller of the two values. Respecting the receive window is flow control, and raising and lowering the congestion window is congestion control.

Slow start and congestion avoidance

These are the two phases in which TCP grows the congestion window. Slow start begins with a very small congestion window and grows it each time an acknowledgment arrives, so the window doubles with every round trip. Despite the name, the window grows fast; it is called slow start because the sender does not send a lot from the beginning. When the congestion window reaches the slow start threshold (ssthresh), TCP moves to congestion avoidance and grows it by only one segment per round trip. When a loss is detected, TCP concludes that the network is congested, lowers the threshold, and reduces the window: by about half if the same acknowledgment arrived three times in a row, and down to one segment if it was a timeout (even if the window at the start of the connection was larger than that). This explanation is the basic method of the standard (RFC 5681), and the algorithm an operating system actually uses may differ in how much it grows and shrinks the window.

TCP and UDP

The two main ways of transporting data on the internet. TCP establishes a connection before communication, guarantees that data arrives complete and in order, and sends data again if there is a problem. UDP sends data without a connection and does not guarantee arrival or order, but in return it is lightweight and fast. HTTP/1.1, HTTP/2, and database connections use TCP, and real-time video and voice and DNS mainly use UDP.

Try these

• Default settings — The window doubles until it reaches the threshold, then grows by one, and when a loss is detected it is cut in half.
• The same loss, detected by a timeout — It starts again from 1 segment, not from half.
• With no loss — It doubles up to the threshold, and after that keeps growing by one.
• When the receiver is slow — Even if the congestion window grows, it sends only up to the receive window of 6 segments.
• With a very large threshold — Slow start keeps doubling until it meets a loss.
• A loss with a receive window of 12 segments — Half is taken not of the congestion window but of the amount actually sent.

Running with the default settings

1. Round trip 1: the congestion window is 1 segment and the receive window is 64 segments. It sends 1 segment, the amount of the congestion window, which is the smaller of the two. Right after the connection is established, it does not know how much the network can handle, so it starts from 1 segment. The congestion window is smaller than the slow start threshold of 8 segments, so this is slow start. The window grows by 1 segment each time an ACK comes back. It receives 1 ACK, and the window becomes 2 segments. That is a doubling in one round trip.
2. Round trip 2: the congestion window is 2 segments and the receive window is 64 segments. The smaller is the congestion window, so it sends 2 segments. Slow start continues. It receives 2 ACKs, and the window becomes 4 segments. That is a doubling in one round trip.
3. Round trip 3: the congestion window is 4 segments and the receive window is 64 segments. The smaller is the congestion window, so it sends 4 segments. Slow start continues. It receives 4 ACKs, and the window becomes 8 segments. That is a doubling in one round trip.
4. Round trip 4: the congestion window is 8 segments and the receive window is 64 segments. The smaller is the congestion window, so it sends 8 segments. The congestion window is at least the slow start threshold of 8 segments, so this is congestion avoidance. Now the window grows by 1 segment only after as many ACKs as the congestion window have arrived. In this round trip it becomes 9 segments. It grows by 1 segment per round trip, carefully.
5. Round trip 5: the congestion window is 9 segments and the receive window is 64 segments. The smaller is the congestion window, so it sends 9 segments. Congestion avoidance continues. In this round trip it becomes 10 segments.
6. Round trip 6: the congestion window is 10 segments and the receive window is 64 segments. The smaller is the congestion window, so it sends 10 segments. But the first segment was lost on the way. For every segment that arrives after it, the receiver repeats the same ACK, which says "I am still waiting for the segment before this one."
7. Three duplicate ACKs have arrived. The sender does not wait for the timer and sends the lost segment again right away (fast retransmit). It treats the loss as a sign that the network is congested. It lowers the slow start threshold to 5 segments, half of the 10 segments it just sent, and it also reduces the congestion window to 5 segments. ACKs are still arriving, so it concludes that the path is not completely blocked and does not lower the window all the way to 1 segment.
8. Round trip 7: the congestion window is 5 segments and the receive window is 64 segments. The smaller is the congestion window, so it sends 5 segments. The congestion window is at least the slow start threshold of 5 segments, so this is congestion avoidance. Now the window grows by 1 segment only after as many ACKs as the congestion window have arrived. In this round trip it becomes 6 segments. It grows by 1 segment per round trip, carefully.
9. Round trip 8: the congestion window is 6 segments and the receive window is 64 segments. The smaller is the congestion window, so it sends 6 segments. Congestion avoidance continues. In this round trip it becomes 7 segments.
10. Round trip 9: the congestion window is 7 segments and the receive window is 64 segments. The smaller is the congestion window, so it sends 7 segments. Congestion avoidance continues. In this round trip it becomes 8 segments.
11. Round trip 10: the congestion window is 8 segments and the receive window is 64 segments. The smaller is the congestion window, so it sends 8 segments. Congestion avoidance continues. In this round trip it becomes 9 segments.
12. Round trip 11: the congestion window is 9 segments and the receive window is 64 segments. The smaller is the congestion window, so it sends 9 segments. Congestion avoidance continues. In this round trip it becomes 10 segments.
13. Round trip 12: the congestion window is 10 segments and the receive window is 64 segments. The smaller is the congestion window, so it sends 10 segments. Congestion avoidance continues. In this round trip it becomes 11 segments.

Result of this run

• When the loss is detected by three duplicate ACKs, the slow start threshold and the congestion window are reduced to half of the amount sent (5 segments), and the window grows by one again from there. It grows, and when it meets a loss it is reduced, over and over, so if you draw the amount sent, it forms a sawtooth.
• In this setting, the receive window was always ample, so the congestion window decided the amount sent.

Code

// The rules of RFC 5681, expressed in units of segments. This is done by TCP in the operating system, not by the program
let cwnd = 1;                  // congestion window: the amount the sender estimates the network can handle. It starts at 1 segment
let ssthresh = 8;              // slow start threshold: grow fast up to here, and carefully beyond it
let acked = 0;                 // number of ACKs counted in congestion avoidance
const rwnd = 64;               // receive window: the room the receiver reported in its ACKs

function canSend() {           // the amount that can be sent at one time
  return Math.min(cwnd, rwnd); // the smaller of the two
}
function onAck() {             // each time one ACK arrives for new data
  if (cwnd < ssthresh) cwnd += 1;                        // slow start: doubles in one round trip
  else if (++acked >= cwnd) { acked = 0; cwnd += 1; }    // congestion avoidance: 1 segment per round trip
}
function onThreeDuplicateAcks(flight) {   // the same ACK three times = one segment was lost
  ssthresh = Math.max(Math.floor(flight / 2), 2);       // half of the amount sent
  cwnd = ssthresh;  acked = 0;                           // continue from half
}

Assumptions of this model

This is a model of the concept that follows the rules of RFC 5681, not the result of running a real TCP. The congestion window inside an operating system cannot be inspected or controlled from a program, so it could not be checked against execution. Instead, the numbers of the model were checked in 153 settings against a separate calculation that applies the same rules one ACK at a time (the model calculates with formulas in units of round trips). The rules that were reproduced are: the amount that can be sent is the smaller of the two windows (RFC 5681 section 3.1); equation (2) for slow start and the recommended method for congestion avoidance (section 3.1); equation (4), which gives the slow start threshold after a loss (section 3.1); 1 segment after a timeout (section 3.1); and the value at the end of fast recovery after three duplicate ACKs (section 3.2). Simplifications: sizes are counted in segments, not in bytes, and halves are rounded down. The initial congestion window is set to 1 segment (the standard allows an upper limit of 2 to 4 segments, and recent operating systems sometimes use 10 segments, RFC 6928). It assumes that one ACK arrives for every segment (in practice, one ACK is sometimes sent for every two segments), it assumes that the receive window does not change (in practice, it changes with every ACK according to how fast the receiving program reads), and it omits the temporary inflation of the window during fast recovery. Setting the initial slow start threshold large enough is the recommendation of the standard, and 8 and 16 are values chosen to make the changes visible. While the sender was blocked by the receive window, the model still grew the congestion window by the formula, but real implementations sometimes do not grow it while they are not using all of it (RFC 7661 sets this rule for the case in which the application has little to send and does not use the whole congestion window). The default algorithm of Linux and Windows is CUBIC (RFC 9438), so its growth curve and its reduction ratio differ from this model.

Previous lab: TCP and UDP · Next lab: HTTP messages and status codes