Entropy of a discrete random variable with possible values {𝑥1,𝑥2,…,𝑥𝑛}…
2017
Entropy of a discrete random variable with possible values {𝑥1,𝑥2,…,𝑥𝑛} and probability density function 𝑃(𝑋) is
\(H(X) = \: – \sum\limits_{i=1}^{n}P(x_i) \log _bP(x_i)\)
The value of 𝑏 gives the units of entropy. The unit for 𝑏 = 10 is
- A.
bits
- B.
bann
- C.
nats
- D.
deca
Attempted by 63 students.
Show answer & explanation
Correct answer: B
Answer: The unit for base 10 is the ban (also called the hartley or decimal digit).
Explanation: The base of the logarithm in the entropy formula determines the unit of information. Common mappings are:
Base 2 → bits
Base e (natural log) → nats
Base 10 → bans (also called hartleys or decimal digits)
Useful conversions:
1 ban ≈ log2(10) ≈ 3.3219 bits
1 nat ≈ log2(e) ≈ 1.4427 bits
Therefore, when b = 10 in the entropy formula, the unit is the ban (hartley).
Loading lesson…