How can teachers improve the reliability and validity of assessments based on…

Concept

Reliability is the consistency of scores across raters or occasions, whereas validity is the extent to which score interpretations represent the intended learning construct.

Because open-ended responses require human judgement, scoring rules must control rater variation while covering the knowledge or skill the task is meant to assess.

Application

A clear rubric states the criteria and performance levels before scoring, and coding categories place similar responses into the same defined classes. Using this common framework makes raters apply the same standards and keeps scoring aligned with the intended construct.

Contrast

• A grammar-only focus narrows scoring to language form and can omit relevant content, reasoning, or evidence.

• Making a prompt easier changes task difficulty; it does not standardize how the resulting responses are judged.

• Permitting several response formats may improve access or expression, but it requires rather than replaces common scoring criteria.

Cross-check and result

If two raters use the same explicit rubric and coding definitions, their classifications can be compared for agreement, while each criterion can be checked against the intended learning outcome. Therefore, constructing clear scoring rubrics and coding categories is the best-supported method.