Given a simple image of size 10 × 10 whose histogram models the symbol…
2014
Given a simple image of size 10 × 10 whose histogram models the symbol probabilities and is given by
p1p2p3p4abcdThe first order estimate of image entropy is maximum when
- A.
\(a = 0, b = 0, c = 0, d = 1\) - B.
\(a=\frac{1}{2}, b=\frac{1}{2}, c=0, d=0\) - C.
\(a=\frac{1}{3}, b=\frac{1}{3}, c=\frac{1}{3}, d=0\) - D.
\(a=\frac{1}{4}, b=\frac{1}{4}, c=\frac{1}{4}, d=\frac{1}{4}\)
Attempted by 118 students.
Show answer & explanation
Correct answer: D
Answer: The entropy is maximized when a = b = c = d = 1/4.
Reasoning: For probabilities a, b, c, d the first-order entropy is H = - (a log a + b log b + c log c + d log d), subject to a + b + c + d = 1.
Use a Lagrange multiplier λ to enforce the constraint: consider L = -Σ p_i log p_i + λ(Σ p_i - 1).
Differentiate with respect to each probability p_i (where p_i is a, b, c, or d): ∂L/∂p_i = - (log p_i + 1) + λ = 0. This implies log p_i is the same for every i, so each p_i is equal.
With four symbols the common value is p_i = 1/4, so a = b = c = d = 1/4.
The maximum entropy value is H = -4*(1/4) log(1/4) = log 4. If logs are base 2, this equals 2 bits.
Therefore the first-order estimate of image entropy is maximized when all four symbol probabilities are equal: a = b = c = d = 1/4.