A measure of consistency among observations of the same situation made by…

Concept

Reliability concerns consistency of measurement, whereas validity concerns whether a measurement supports its intended interpretation. A reliability label is chosen by identifying what is being compared: occasions, item sets, or raters.

Application

1. The observations concern the same situation, so the target being assessed is held constant.

2. Different people make the observations, so the comparison is among raters rather than occasions or item sets.

3. Consistency or agreement among ratings made by different observers is inter-rater reliability.

Contrast

• Construct validity examines whether observed scores represent the intended theoretical construct.

• Internal validity examines whether a causal conclusion is protected from confounding explanations.

• Split-half reliability compares two equivalent halves of one measurement instrument.

Cross-check

Repeated measurement by the same source would point to stability over time, and comparison of two halves would point to split-half reliability. Here the changing element is the observer, which confirms inter-rater reliability.

Result

Therefore, the measure described is inter-rater reliability.