TCP Protocol - Deep Dive Study Guide

TCP (Transmission Control Protocol) is a connection-oriented, reliable, byte-stream protocol that operates at the Transport Layer (Layer 4) of the OSI model.

Key Characteristics:

Reliable: Guarantees delivery of data in order, without loss or duplication
Connection-oriented: Establishes connection before data transfer (handshake)
Flow control: Prevents overwhelming the receiver
Congestion control: Prevents overwhelming the network
Full-duplex: Bidirectional communication
Byte-stream: Treats data as a continuous stream (no message boundaries)

TCP vs UDP:

Aspect	TCP	UDP
Connection	Connection-oriented	Connectionless
Reliability	Guaranteed delivery	Best effort (no guarantees)
Ordering	In-order delivery	No ordering
Speed	Slower (overhead)	Faster (minimal overhead)
Use cases	HTTP, FTP, email, SSH	DNS, streaming, gaming, VoIP

TCP Connection Lifecycle

1. Three-Way Handshake (Connection Establishment)

sequenceDiagram participant Client participant Server Note over Client: CLOSED Client->>+Server: SYN
(seq=x, SYN flag=1) Note over Client: SYN_SENT Note over Server: SYN_RECEIVED Server-->>-Client: SYN-ACK
(seq=y, ack=x+1, SYN+ACK flags=1) Client->>Server: ACK
(ack=y+1) Note over Client: ESTABLISHED Note over Server: ESTABLISHED rect rgb(46, 52, 64) Note over Client,Server: CONNECTION ESTABLISHED
Ready for data transfer end

Steps:

SYN: Client sends SYN packet with initial sequence number (ISN)
SYN-ACK: Server responds with SYN-ACK, acknowledging client's ISN and sending its own ISN
ACK: Client acknowledges server's ISN

Purpose: Synchronize sequence numbers, establish connection parameters (window size, MSS)

Interview Tip: Why three steps? Two-way handshake can't handle delayed duplicate SYN packets. Three-way prevents old duplicate SYNs from opening unwanted connections.

2. Four-Way Handshake (Connection Termination)

sequenceDiagram participant Client participant Server Note over Client: ESTABLISHED Note over Server: ESTABLISHED Client->>+Server: FIN
(seq=x, FIN flag=1) Note over Client: FIN_WAIT_1 Server-->>-Client: ACK
(ack=x+1) Note over Client: FIN_WAIT_2 Note over Server: CLOSE_WAIT Note over Server: Server finishes sending
remaining data Server->>Client: FIN
(seq=y, FIN flag=1) Note over Server: LAST_ACK Client->>Server: ACK
(ack=y+1) Note over Client: TIME_WAIT Note over Client: Wait 2xMSL
(~2-4 minutes) rect rgb(59, 66, 82) Note over Client: CLOSED Note over Server: CLOSED Note over Client,Server: CONNECTION TERMINATED end

Steps:

FIN: Client sends FIN to close its side of the connection
ACK: Server acknowledges the FIN
FIN: Server sends its own FIN when ready to close
ACK: Client acknowledges server's FIN

Note: Client enters TIME_WAIT state (2xMSL) to handle delayed packets before fully closing.

TCP Reliability Mechanisms

1. Sequence Numbers and Acknowledgments

How it works:

Every byte of data has a sequence number
Receiver sends ACK with next expected sequence number
ACKs are cumulative (ACK 1000 means bytes 0-999 received)

Example:

Client sends: SEQ=1000, Data="Hello" (5 bytes)
Server ACKs:  ACK=1005 (expecting byte 1005 next)

Client sends: SEQ=1005, Data="World" (5 bytes)
Server ACKs:  ACK=1010

# If packet lost:
Client sends: SEQ=1010, Data="!!!" (3 bytes)
[packet lost, server doesn't receive it]

Client sends: SEQ=1013, Data="Next" (4 bytes)
Server ACKs:  ACK=1010 (still waiting for byte 1010-1012)

2. Retransmission and Timeouts

Retransmission Timer (RTO - Retransmission Timeout)

TCP uses adaptive timeout based on Round-Trip Time (RTT):

Measure RTT: Time from sending data to receiving ACK
Calculate SRTT (Smoothed RTT): Exponential weighted moving average
Calculate RTO: RTO = SRTT + 4 x RTTVAR

Karn's Algorithm

Don't use RTT samples from retransmitted segments (ambiguous - which transmission was ACKed?).

Fast Retransmit

Don't wait for timeout if you get 3 duplicate ACKs:

Receive 3 duplicate ACKs for same sequence number
Immediately retransmit the missing segment
Don't wait for RTO timeout (faster recovery)

Example: Fast Retransmit

Client sends: SEQ=1000, SEQ=1500, SEQ=2000, SEQ=2500
Server receives: 1000, 1500 (lost), 2000, 2500

Server sends: ACK=1500, ACK=1500, ACK=1500, ACK=1500
              (duplicate ACKs because still waiting for 1500)

Client receives 3 duplicate ACKs - immediately retransmits SEQ=1500
(Don't wait for timeout!)

Interview Tip: Fast Retransmit reduces latency significantly compared to waiting for RTO timeout.

TCP Flow Control - Sliding Window

Sliding Window Protocol

Purpose: Prevent sender from overwhelming receiver's buffer.

How it works:

Receiver advertises receive window (rwnd) in every ACK
Sender can send up to rwnd bytes without waiting for ACK
Window "slides" forward as ACKs are received

graph LR subgraph Initial["Initial State (rwnd = 4000 bytes)"] A1["0-999
ACKed"] A2["1000-2999
Sent
Awaiting ACK"] A3["3000-4999
Can Send"] A4["5000+
Blocked"] end subgraph After["After ACK=2000 (Window Slides Forward)"] B1["0-1999
ACKed"] B2["2000-2999
Sent
Awaiting ACK"] B3["3000-5999
Can Send"] B4["6000+
Blocked"] end A1 --> A2 A2 --> A3 A3 --> A4 B1 --> B2 B2 --> B3 B3 --> B4 Initial -->|"Receive ACK=2000
Window slides"| After style A1 fill:#3b4252,stroke:#a3be8c,stroke-width:2px,color:#e0e0e0 style A2 fill:#3b4252,stroke:#ebcb8b,stroke-width:2px,color:#e0e0e0 style A3 fill:#3b4252,stroke:#88c0d0,stroke-width:2px,color:#e0e0e0 style A4 fill:#3b4252,stroke:#bf616a,stroke-width:2px,color:#e0e0e0 style B1 fill:#3b4252,stroke:#a3be8c,stroke-width:2px,color:#e0e0e0 style B2 fill:#3b4252,stroke:#ebcb8b,stroke-width:2px,color:#e0e0e0 style B3 fill:#3b4252,stroke:#88c0d0,stroke-width:2px,color:#e0e0e0 style B4 fill:#3b4252,stroke:#bf616a,stroke-width:2px,color:#e0e0e0

Usable Window: Initially 2000 bytes (3000-4999), after ACK: 3000 bytes (3000-5999)

Example: Flow Control in Action

# Initial state
Receiver buffer: 10KB free - advertises rwnd=10240
Sender sends: 5KB of data

# Receiver processes 2KB, has 7KB in buffer
Receiver ACKs with: rwnd=3072 (10KB - 7KB)
Sender can now send up to 3072 bytes

# Receiver processes all buffered data
Receiver ACKs with: rwnd=10240 (buffer empty)
Sender can send full window again

Interview Tip: Sliding window enables pipelining (send multiple packets before waiting for ACKs), dramatically improving throughput compared to stop-and-wait.

TCP Congestion Control

Overview: Why Congestion Control?

Flow control (sliding window) handles receiver capacity. Congestion control handles network capacity - preventing routers from being overwhelmed.

Congestion Window (cwnd)

Maintained by sender (not advertised by receiver)
Limits how much data sender can have in-flight
Effective window = min(cwnd, rwnd)

1. Slow Start

Purpose: Quickly find available bandwidth without causing congestion.

How it works:

Start with small cwnd (typically 1 MSS, modern: 10 MSS)
Double cwnd for each ACK (exponential growth)
Continue until:
- Reach ssthresh (slow start threshold), OR
- Detect loss (timeout or duplicate ACKs)

Example: Slow Start Growth

RTT 1: cwnd = 1 MSS  - send 1 packet
       Receive 1 ACK - cwnd = 2 MSS

RTT 2: cwnd = 2 MSS  - send 2 packets
       Receive 2 ACKs - cwnd = 4 MSS

RTT 3: cwnd = 4 MSS  - send 4 packets
       Receive 4 ACKs - cwnd = 8 MSS

RTT 4: cwnd = 8 MSS  - send 8 packets
       Receive 8 ACKs - cwnd = 16 MSS

# Exponential growth: 1 - 2 - 4 - 8 - 16 - 32 - 64 ...

Why "Slow Start"? Ironically named because it's actually aggressive exponential growth. Called "slow" compared to sending at full speed immediately.

2. Congestion Avoidance

Purpose: Increase bandwidth usage cautiously after slow start.

How it works:

Triggered when cwnd reaches ssthresh
Increase cwnd by 1 MSS per RTT (additive increase)
Linear growth instead of exponential

Example: Congestion Avoidance

Assume ssthresh = 16 MSS

RTT 1: cwnd = 16 MSS - cwnd = 17 MSS (increase by 1)
RTT 2: cwnd = 17 MSS - cwnd = 18 MSS
RTT 3: cwnd = 18 MSS - cwnd = 19 MSS
RTT 4: cwnd = 19 MSS - cwnd = 20 MSS

# Linear growth: 16 - 17 - 18 - 19 - 20 ...

3. Multiplicative Decrease (Congestion Detected)

When Loss Detected:

Option A: Timeout (severe congestion)

Set ssthresh = cwnd / 2
Set cwnd = 1 MSS (restart slow start)
This is the "exponential backoff" part!

Option B: 3 Duplicate ACKs (mild congestion)

Fast Retransmit: Immediately retransmit missing segment
Fast Recovery:
- Set ssthresh = cwnd / 2
- Set cwnd = ssthresh (don't drop to 1)
- Enter congestion avoidance directly

Exponential Backoff (Congestion Control):

When timeout occurs, TCP halves the congestion window (multiplicative decrease) and may double the RTO for retransmissions. This is the exponential backoff mechanism that prevents overwhelming a congested network.

Example: Multiplicative Decrease

# Scenario 1: Timeout
cwnd = 32 MSS
[Timeout occurs]
ssthresh = 32 / 2 = 16 MSS
cwnd = 1 MSS
- Restart slow start, grow to 16, then congestion avoidance

# Scenario 2: 3 Duplicate ACKs
cwnd = 32 MSS
[3 duplicate ACKs received]
ssthresh = 32 / 2 = 16 MSS
cwnd = 16 MSS
- Fast recovery, enter congestion avoidance immediately

AIMD: Additive Increase, Multiplicative Decrease

The core principle of TCP congestion control:

Additive Increase: Increase cwnd by 1 MSS per RTT (linear growth)
Multiplicative Decrease: Halve cwnd on loss (exponential backoff)

Result: TCP "probes" for available bandwidth, backing off when congestion detected.

graph LR subgraph Timeline["TCP Congestion Window Evolution"] A[cwnd=1
Start] -->|"Slow Start
Exponential"| B[cwnd=2] B --> C[cwnd=4] C --> D[cwnd=8] D --> E[cwnd=16
ssthresh] E -->|"Congestion
Avoidance
Linear"| F[cwnd=17] F --> G[cwnd=18] G --> H[cwnd=19] H --> I[cwnd=20] I --> J["Loss!
3 dup ACKs"] J -->|"Fast Recovery
cwnd = cwnd/2"| K[cwnd=10] K -->|"Congestion
Avoidance"| L[cwnd=11] L --> M[cwnd=12] M --> N["Timeout!"] N -->|"Multiplicative
Decrease
cwnd=1"| O[cwnd=1
Restart] O -->|"Slow Start
Again"| P[cwnd=2] end style A fill:#3b4252,stroke:#a3be8c,stroke-width:2px,color:#e0e0e0 style E fill:#3b4252,stroke:#ebcb8b,stroke-width:2px,color:#e0e0e0 style J fill:#3b4252,stroke:#bf616a,stroke-width:2px,color:#e0e0e0 style N fill:#3b4252,stroke:#bf616a,stroke-width:2px,color:#e0e0e0 style O fill:#3b4252,stroke:#a3be8c,stroke-width:2px,color:#e0e0e0

Pattern: Slow Start (exponential) - Congestion Avoidance (linear) - Loss (decrease) - Repeat

Advanced TCP Algorithms

TCP Tahoe vs Reno vs NewReno vs CUBIC

Algorithm	Loss Detection	Response	Notes
Tahoe	Timeout or 3 dup ACKs	cwnd = 1, restart slow start	Original, very conservative
Reno	Timeout: cwnd = 1 3 dup ACKs: Fast Recovery	Fast Retransmit + Fast Recovery	Most common classic algorithm
NewReno	Like Reno, better multi-loss handling	Improved fast recovery	Handles multiple losses in one window
CUBIC	Like Reno	Cubic function for cwnd growth	Default in Linux, better for high-bandwidth
BBR	Doesn't rely on loss	Model-based, optimizes for throughput + latency	Google's algorithm, used in YouTube

TCP Header Format

 0                   1                   2                   3
 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1 2 3 4 5 6 7 8 9 0 1
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|          Source Port          |       Destination Port        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                        Sequence Number                        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Acknowledgment Number                      |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|  Data |       |C|E|U|A|P|R|S|F|                               |
| Offset| Rsrvd |W|C|R|C|S|S|Y|I|            Window             |
|       |       |R|E|G|K|H|T|N|N|                               |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|           Checksum            |         Urgent Pointer        |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                    Options                    |    Padding    |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+
|                             Data                              |
+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+-+

Key Fields:

Sequence Number: Identifies byte position in data stream
ACK Number: Next expected byte from other side
Window: Receive window size (flow control)
Flags:
- SYN: Synchronize (connection setup)
- ACK: Acknowledgment valid
- FIN: Finish (connection teardown)
- RST: Reset (abort connection)
- PSH: Push (deliver data immediately)
- URG: Urgent pointer valid
Checksum: Error detection
Options: MSS, window scaling, timestamps, SACK

Interview Key Points

Common Interview Questions & Answers

Q: How does TCP ensure reliability?

Sequence numbers: Track every byte
Acknowledgments: Receiver confirms receipt
Retransmission: Resend lost packets (timeout or fast retransmit)
Checksums: Detect corrupted data
Ordering: Reorder out-of-order packets

Q: Explain TCP congestion control

Slow Start: Exponentially increase cwnd (doubling)
Congestion Avoidance: Linearly increase cwnd (AIMD)
Loss Detection: Timeout or 3 duplicate ACKs
Multiplicative Decrease: Halve cwnd on loss
Goal: Find optimal sending rate without overwhelming network

Q: What is the sliding window?

Purpose: Flow control (prevent overwhelming receiver)
Receiver advertises available buffer space (rwnd)
Sender can send up to rwnd bytes without ACK
Window slides forward as ACKs arrive
Enables pipelining (better throughput than stop-and-wait)

Q: Flow control vs congestion control?

Flow control (rwnd): Receiver tells sender how much it can accept
Congestion control (cwnd): Sender infers network capacity from loss
Effective window = min(rwnd, cwnd)

Q: Why 3-way handshake, not 2-way?

Prevents old duplicate SYN packets from opening unwanted connections
Both sides confirm each other's initial sequence numbers
Ensures both sides are ready for communication

Q: What is TIME_WAIT state?

Client enters TIME_WAIT after sending final ACK in 4-way handshake
Waits 2xMSL (Maximum Segment Lifetime, typically 60-120 seconds)
Ensures:
- Final ACK is received (can retransmit if needed)
- Old duplicate packets expire before port reuse

Summary

TCP Key Takeaways

Reliability: Sequence numbers, ACKs, retransmission, checksums
Flow Control: Sliding window (rwnd) prevents receiver overload
Congestion Control: Slow start + AIMD prevents network overload
Exponential Backoff: Halve cwnd on loss (multiplicative decrease)
Fast Retransmit: 3 duplicate ACKs trigger immediate retransmission
Connection-oriented: 3-way handshake (setup), 4-way handshake (teardown)
In-order delivery: Reorders packets using sequence numbers
Full-duplex: Bidirectional communication

The Brilliance of TCP Design

TCP elegantly solves multiple hard problems:

Reliability over unreliable networks (Internet)
Fairness among competing flows (AIMD converges to fair shares)
Efficiency through pipelining (sliding window)
Adaptability to varying network conditions (congestion control)
Simplicity from applications' perspective (just a byte stream)

Why TCP is still dominant after 40+ years: Its fundamental design principles (end-to-end reliability, congestion control, flow control) remain sound. Modern variants (CUBIC, BBR) improve performance while keeping the core mechanisms.