Consider a discrete memoryless channel and assume that H(x) is the amount of…
2016
Consider a discrete memoryless channel and assume that H(x) is the amount of information per symbol at the input of the channel; H(y) is the amount of information per symbol at the output of the channel; H(x|y) is the amount of uncertainty remaining on x knowing y; and I (x; y) is the information transmission.
Which of the following does not define the channel capacity of a discrete memoryless channel ?
- A.
max I(x;y);
p(x)
- B.
max [H(y)-H(y ∣ x)];
p(x)
- C.
max [H(x)-H(x ∣ y)];
p(x)
- D.
max H(x ∣y);
p(x)
Attempted by 49 students.
Show answer & explanation
Correct answer: D
Answer: The expression that does not define channel capacity is max H(x|y) over p(x).
Explanation:
Definition: Mutual information I(X;Y) measures the amount of information about X obtained from Y.
Equivalent forms: I(X;Y) = H(Y) - H(Y|X) = H(X) - H(X|Y).
Channel capacity: C = max_{p(x)} I(X;Y). Therefore maximizing I(X;Y), maximizing H(Y)-H(Y|X), or maximizing H(X)-H(X|Y) (all over p(x)) are valid expressions for capacity.
Why max H(X|Y) is wrong: H(X|Y) is the residual uncertainty about the input after observing the output. Maximizing this quantity increases uncertainty remaining, which is the opposite of maximizing transmitted information.
Counterexample: For a noiseless channel Y = X, H(X|Y) = 0 for any input distribution, but the capacity equals the maximum H(X) (which can be positive). Thus max H(X|Y) would give 0 and not the capacity.
Conclusion: The first three expressions (max I(X;Y), max [H(Y)-H(Y|X)], max [H(X)-H(X|Y)] over p(x)) correctly define channel capacity. Maximizing H(X|Y) does not.