The degree to which two or more observers agree in their coding and scoring of…

Concept — In observational research two different kinds of construct describe the quality of a behavioural record. One kind names a source of error: something that distorts what gets written down, arising either in the person doing the recording or in the people being watched. The other kind names a consistency index: a quantified statement of how closely independent measurements of the same event correspond. A statement about how far several raters agree when scoring the same events is, by definition, a consistency index computed across raters.

Application — Here several observers code and score the same stream of behaviour, and the question asks what the extent of their agreement relates to. Agreement measured between independent raters on the same events is inter-observer reliability (also called inter-rater reliability). It is reported as percentage agreement or as a chance-corrected coefficient such as Cohen’s kappa. A high value shows that the codes depend little on which observer happened to be watching; such consistency is a necessary condition for a trustworthy record but not a sufficient one, because observers trained together can still share the same systematic bias.

Contrast — the other constructs named here describe something different:

• Observer bias — a systematic distortion produced when a recorder’s expectations or preferences shape how an event is coded; it is a source of error inside one record, not a measure of agreement between records.

• Observer drift — a gradual change in how one recorder applies the coding rules across a session; it concerns instability within a single rater over time, not concordance between raters.

• Subject reactivity — a change in the participants’ own behaviour caused by their awareness of being observed; it alters the behaviour that is available to be coded and says nothing about how observers score it.

Result — the construct that expresses the degree of agreement among two or more observers coding and scoring the same behaviour is inter-observer reliability.