Given a dataset with K binary-valued attributes (where K > 2) for a two-class…
2024
Given a dataset with K binary-valued attributes (where K > 2) for a two-class classification task, the number of parameters to be estimated for learning a naïve Bayes classifier is
Answer: B. 2K + 1 — A naïve Bayes classifier stores two kinds of numbers: the class prior P(C), and for every attribute the class-conditional distribution P(Xj | C). The naïve…
- A.
2K + 1
- B.
2K + 1
- C.
2K+1 + 1
- D.
K2 + 1
Attempted by 50 students.
Show answer & explanation
Correct answer: B
A naïve Bayes classifier stores two kinds of numbers: the class prior P(C), and for every attribute the class-conditional distribution P(Xj | C). The naïve assumption of conditional independence means each attribute carries its own distribution within each class instead of one distribution over all attribute combinations. So the parameter count is (free parameters of the prior) + (number of classes × number of attributes × free parameters per attribute distribution). A distribution over q outcomes has q − 1 free parameters, because the last probability is fixed by the others.
Classes: q = 2, so the prior contributes 2 − 1 = 1 free parameter, say P(C = 1); then P(C = 0) = 1 − P(C = 1) follows.
Attributes: each Xj takes two values, so P(Xj = 1 | C = c) is 1 free parameter; then P(Xj = 0 | C = c) = 1 − P(Xj = 1 | C = c) follows.
There are K attributes and 2 classes, so the class-conditional parameters number 2 × K = 2K.
Total = 1 (prior) + 2K (class-conditional) = 2K + 1.
Cross-check with K = 3, listing the parameters:
P(C = 1); P(X1 = 1 | C = 0), P(X2 = 1 | C = 0), P(X3 = 1 | C = 0); P(X1 = 1 | C = 1), P(X2 = 1 | C = 1), P(X3 = 1 | C = 1). That is 1 + 6 = 7 parameters, and 2K + 1 gives 2(3) + 1 = 7.
Which model each of the other counts would describe:
Parameter count | Model it would describe |
|---|---|
2K + 1 | One parameter for every joint assignment of the K binary attributes, i.e. a full joint table with no independence assumption. |
2K + 1 | One Bernoulli parameter per attribute within each class, plus the class prior. |
2K+1 + 1 | A separate full joint table over the K binary attributes for each of the two classes. |
K2 + 1 | One parameter for every ordered pair of attributes, i.e. pairwise interactions, which conditional independence never estimates. |
Hence the number of parameters to be estimated is 2K + 1.
Explore the full course: Iocl Engineers Officer Grade A Paper 1