Good evaluation of written material should not be based on :

Concept — an evaluation is valid only when it rests on construct-relevant evidence. Evaluation is a value judgement passed on a performance against stated criteria. Measurement theory admits a criterion into that judgement only when the evidence it supplies belongs to the trait the assessment actually intends to measure. Any part of the score that is produced by something else — the polish of a candidate’s language, the length of the answer, the neatness of the hand — is called construct-irrelevant variance, and it is a recognised threat to the validity of the judgement.

Application — what the evaluation of written material is meant to measure. When a script written in a subject examination is assessed, the intended construct is the candidate’s command of the subject. Three of the offered criteria supply evidence that belongs to that construct:

• Comprehension of subject — whether the script shows that the ideas have actually been understood.

• Logical presentation — whether the ideas are ordered, connected and built into an argument.

• Ability to reproduce whatever is read — whether the studied content has been retained and restated accurately.

Linguistic expression is different in kind. Vocabulary, grammar and style measure a candidate’s facility with the language in which the script happens to be written, not command of the subject being examined. Two candidates with identical subject command but different language backgrounds would be separated by that criterion alone, and the separation would carry no information about the trait under assessment. Basing the judgement on it therefore imports construct-irrelevant variance into the score.

Cross-check — substitute each value back into the stem.

• “Based on comprehension of subject”: what is scored is grasp of the content, which is the trait being examined.

• “Based on logical presentation”: what is scored is the organisation of the subject matter, which reflects command of that matter.

• “Based on ability to reproduce whatever is read”: what is scored is retention of the studied content. It is the lowest-order kind of subject evidence, which is why many candidates are drawn to this value, but it is still evidence about the subject rather than about the medium.

• “Based on linguistic expression”: what is scored is skill in the language itself, which sits outside the trait the examination set out to measure.

Result. Only the language criterion falls outside the trait the examination set out to measure, so good evaluation of written material should not be based on Linguistic expression.

Source note. This item was set with the same four values in West Bengal SET, General Paper I, 6 December 2015 (question 5), and the West Bengal College Service Commission’s published final answer key marks “linguistic expression” for it. Some coaching compilations of the UGC NET June 2009 paper print a different value; no official key was released for that 2009 paper, so the commission’s final key is the authoritative reading and is the one followed here.