Research Validity and Reliability for Teaching Exams: Notes with Worked Examples

Validity, reliability, objectivity and generalisability look similar in revision, and teaching exams hide the tested distinction inside a study or a measurement scenario rather than asking for a definition. Validity asks whether the evidence supports the interpretation you want to make. Reliability asks whether the same measurement would hold steady on a repeat. Objectivity asks whether two independent scorers would award the same mark. Generalisability asks how far the finding travels beyond the group studied. Four questions, not four memorised sentences. CTET, state TETs, UGC NET and teaching-recruitment papers do not share one universal syllabus or pattern, so learn the reasoning before checking a specific exam's current documents.

Research basics: from a classroom concern to an answerable question

Research answers questions through a systematic, logical, empirical process: problem, question, review, design, collection, analysis, conclusion, revised question. Research Process Steps for Teaching Exams walks that sequence stage by stage, with the sampling arithmetic and hypothesis writing. The stages are the easy part. What decides an exam answer is whether the evidence a stage produces can carry the interpretation placed on it, because systematic does not mean correct.

"18 of 30 learners submitted the task" is an observation. "The other 12 lacked motivation" is an inference needing evidence.

Running question: Does a 10-minute retrieval quiz after each science lesson improve four-week recall among Class VIII learners compared with 10 minutes of rereading? The activity is the independent variable, score out of 25 the dependent variable, Class VIII learners of interest the population, and 60 participants the sample.

Validity: does the evidence support the intended interpretation?

Validity concerns evidence interpretation and use, not a permanent label. Instrument validity concerns measurement; study validity includes design and conclusions.

Suppose the 25 one-mark items include 10 factual-recall, 10 concept-application and 5 explanation items. If all cover Week 1 definitions, the intended four-week delayed-recall construct is poorly represented, even with consistent scoring.

Random assignment mainly strengthens causal comparability and internal validity. Random sampling mainly strengthens representativeness and external validity. Neither ensures the other.

Fully worked validity example: the 60-learner retrieval study

The figures below are hypothetical, chosen so that every threat stays visible. Randomly assign 60 Class VIII learners to two groups of 30. Both study the same four-week unit with the same teacher and lesson time. Q uses the last 10 minutes for retrieval; R rereads.

Both start with mean pre-test 12/25. At Week 4, Q averages 19/25 and R averages 16/25.

1. Q's mean gain is 19 - 12 = 7 points.

2. R's mean gain is 16 - 12 = 4 points.

3. The difference in mean gains is 7 - 4 = 3 points.

In this hypothetical study, retrieval produced a three-point larger mean gain. Do not generalise it to all learners or subjects.

If Q is taught in the morning and R after lunch, time of day threatens internal validity. Repeated quiz items weaken construct validity; definitions-only items weaken content validity. One school limits external validity rather than removing it. Score reliability still needs evidence.

Reliability: would the measurement stay consistent?

Reliability means score consistency under stated conditions. Test-retest concerns time, parallel forms equivalent versions, internal consistency related items, and inter-rater reliability scorers. Stability can still measure the wrong construct.

Five learners score [12, 15, 18, 20, 25] on Day 1 and [13, 15, 17, 21, 24] on Day 8. Absolute differences are [1, 0, 1, 1, 1], and both means are 18. Using deviations from 18:

• cross-products sum to 87;

• squared deviations sum to 98 and 80;

• Pearson's r = 87 / sqrt(98 x 80) = 0.9826, approximately 0.98.

The pattern is highly consistent, but five learners cannot establish reliability, and correlation misses systematic bias.

For inter-rater reliability, A records [P, P, R, P, R, P, R, P] and B records [P, R, R, P, R, P, R, P] for eight proposals. They agree on 7, so observed agreement is 7/8 x 100 = 87.5%. Percentage agreement ignores chance; a real study may need kappa. Agreement between independent scorers is also what exams mean by objectivity, which is why essay marking is judged on this figure rather than on test-retest stability.

Validity versus reliability: four exact measurement patterns

Use a reference temperature of 37.0 degrees Celsius.

Reliability is generally necessary but insufficient for a validity claim. B repeats bias. C reaches the reference only as an average, so one reading is undependable.

Temperature only illustrates consistency and bias. Research validity also depends on design, representation, reasoning and use. Consistent scoring cannot remove the study's time-of-day threat or poor coverage.

How teaching exams turn these concepts into questions

Common tasks match definitions, name a type, diagnose a threat, separate random assignment from sampling, interpret a coefficient or agreement, and judge "a reliable instrument is always valid".

Rapid checks:

1. Scores always 12 points above a trusted reference show reliability without validity: stable bias.

2. Agreement between two essay scorers concerns inter-rater reliability.

3. A one-school result that may not transfer raises external validity, not necessarily internal validity.

UGC NET candidates meet this material in Paper 1, and UGC NET Paper 1 vs Paper 2 sets out what each paper carries. Check exact inclusion in CTET, a state TET, UGC NET or a named recruitment paper against its current official syllabus, bulletin or notification. KnowledgeGate's research aptitude bank carries more than 50 practice questions on validity and reliability alone, though not every one of them maps to every teaching exam.

High-frequency traps and the correction for each

Sort with six checks: domain covered, content validity; intended idea measured, construct validity; another explanation, internal validity; transfer elsewhere, external validity; stable over time, test-retest reliability; judges agree, inter-rater reliability.

Running study: repeated items indicate construct validity; time of day, internal validity; one school, external validity; unstable repeat scores, reliability.

Research validity and reliability: the short version and next step

Recall the chain:

1. Ask a clear question.

2. Define variables.

3. Choose a design.

4. Gather evidence consistently.

5. Test validity.

6. Limit the conclusion to the design's support.

For a 30-minute loop, spend 10 minutes rebuilding the matrix, 12 solving six vignettes and 8 logging each wrong option's failed distinction.

Continue concepts with the NTA UGC NET Paper 1 concept course, practise with the UGC NET Paper 1 test series, or browse the UGC NET preparation courses and test series category.