TCP Deep Dive

What TCP Guarantees

TCP (RFC 793, updated by RFC 9293) sits at the transport layer and provides four core guarantees that raw IP does not: connection-oriented communication, reliable delivery, ordered byte arrival, and a byte-stream abstraction that hides packet boundaries from the application.

Handshake, Data Transfer, and Teardown

The 3-Way Handshake

Before any data flows, client and server synchronize sequence numbers via three segments: SYN, SYN-ACK, ACK. This prevents stale connections from being accepted and lets both sides agree on initial sequence numbers (ISNs).

The 4-Way Teardown

Either side can initiate a close. Because TCP is full-duplex, each direction must be shut down independently. The initiator sends FIN, gets ACK, then the other side sends its FIN and receives ACK. This produces the well-known FIN_WAIT_1 → FIN_WAIT_2 → TIME_WAIT progression on the initiating side.

Sequence Numbers, ACKs, and the Sliding Window

Every byte in a TCP stream has a 32-bit sequence number. The receiver advertises a receive window (rwnd) -- the number of bytes it can buffer. The sender must not have more than rwnd bytes outstanding (sent but not yet acknowledged).

Window Scaling (RFC 7323)

The original 16-bit window field caps rwnd at 64 KB -- far too small for high-bandwidth, high-latency links (bandwidth-delay product). The window scale option, negotiated during the handshake, applies a left-shift of up to 14 bits, supporting windows up to ~1 GB.

Cumulative vs Selective ACK

Standard TCP ACKs are cumulative: an ACK number of N means "I have received all bytes up to N-1." This is simple but forces retransmission of everything after a gap. Selective ACK (SACK), defined in RFC 2018, lets the receiver report non-contiguous blocks it has received, so the sender retransmits only the missing segments.

Slow Start, CUBIC, and BBR

Flow control prevents overwhelming the receiver. Congestion control prevents overwhelming the network. The sender maintains a congestion window (cwnd). The effective sending window is min(cwnd, rwnd).

Phases of Congestion Control

CUBIC (Linux default)

CUBIC uses a cubic function of time since the last loss event to set cwnd, making it less aggressive near the previous maximum but more aggressive when far below it. This produces faster recovery on high-BDP links than the classic Reno/NewReno linear increase.

BBR (Bottleneck Bandwidth and RTT)

Developed by Google, BBR does not use packet loss as the primary congestion signal. Instead it estimates the bottleneck bandwidth and the minimum RTT, then paces packets to match. This works much better on links where buffer bloat causes loss-based algorithms to underperform.

States, Options, and Practical Tuning

The 11 TCP States

Important Options and Tuning

TCP's Limits and the Path to QUIC

Head-of-Line Blocking

TCP guarantees ordered delivery within a single byte stream. If segment #5 is lost, segments #6, #7, #8 must wait in the receive buffer even though they arrived fine. For HTTP/2, which multiplexes many logical streams over one TCP connection, a single lost packet blocks all streams. This is TCP-level head-of-line blocking.

HTTP/3 solves this by replacing TCP with QUIC, which runs over UDP and implements its own per-stream reliability. A lost packet on stream B only blocks stream B -- streams A and C continue unaffected.

Socket Programming: Server Lifecycle

Understanding TCP's connection model is essential for server programming. The kernel manages two queues for each listening socket:

// Minimal TCP server in C
int server_fd = socket(AF_INET, SOCK_STREAM, 0);
setsockopt(server_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

struct sockaddr_in addr = {
    .sin_family = AF_INET,
    .sin_port = htons(8080),
    .sin_addr.s_addr = INADDR_ANY
};
bind(server_fd, (struct sockaddr*)&addr, sizeof(addr));
listen(server_fd, 128);  // backlog = 128

while (1) {
    int client_fd = accept(server_fd, NULL, NULL);
    // read/write on client_fd ...
    close(client_fd);
}