Information Theory, Coding, and Error Control

Manual: General · Subject: Networking

Study the limits of reliable communication and the mechanisms used to approach them.

Channel Capacity and Reliable Communication

Shannon's insight

A noisy channel can still support arbitrarily reliable communication if the transmission rate is below capacity and sufficiently strong coding is used. The capacity $C$ is the highest rate at which error probability can be made arbitrarily small in the limit of long codewords.

Important concepts

Entropy

H(X)

Uncertainty in a random variable

Mutual information

I(X;Y)

Information learned about

X

from observing

Y

Channel capacity

C

Maximum reliable communication rate

Bit error rate

Fraction of bits received incorrectly

Forward error correction

Adding redundancy to correct errors without retransmission

Basic coding tradeoffs

Adding redundancy improves error correction but reduces net rate. Practical schemes balance coding gain, decoding complexity, latency, and energy use. In high-delay environments, forward error correction may outperform retransmission-based recovery because round-trip time dominates repair time.

ARQ versus FEC

ARQ

Uses acknowledgments and retransmissions
Simple and efficient when RTT is low and loss is rare
Can increase latency under congestion or long RTT

FEC

Adds redundancy up front
Avoids waiting for retransmissions
Useful when delay is expensive or feedback is unreliable

What does Shannon capacity represent?

The minimum delay possible on any network The maximum rate of reliable communication over a noisy channel The number of packets a router can store The average size of a TCP segment

Why is long codeword length often beneficial in coding theory?

How to think about a coding system

1

Model the channel and its impairments.
2

Choose an error metric such as packet error rate or bit error rate.
3

Select a coding family that matches latency and complexity constraints.
4

Estimate overhead and effective throughput.
5

Evaluate decoding performance under realistic traffic and channel conditions.

💡

Advanced note

In modern systems, coding is often combined with interleaving, adaptive modulation, and cross-layer link adaptation to exploit time-varying channel quality.

Which factor most directly increases retransmission latency?

Lower RTT Higher RTT Higher buffer memory Lower packet size

Neuroanatomy Explorer

Memory Deck

Match the Pairs

Concept Constellation