Table 5

Mean exact agreement and Gwet’s AC2 for each group of observers (researchers, clinicians and students) and for all observers combined (AC2 values adjusted for critical values*)

ObserverInter-observer
Mean % exact agreementGwet’s AC2/classification
Researcher 161.1
Researcher 265.0
Researcher 358.2
Researcher 462.1
Researcher 558.2
Researcher 661.4
Researcher 762.5
Researcher 864.3
Mean1 (SD or 95% CI)61.6 (2.5)0.81 (0.76 to 0.86)
Adjusted AC20.70/Substantial
Clinician 138.6
Clinician 255.0
Clinician 361.8
Clinician 453.6
Clinician 560.4
Clinician 663.9
Clinician 765.4
Clinician 862.1
Mean2 (SD or 95% CI)57.6 (8.7)0.80 (0.77 to 0.84)
Adjusted AC20.69/Substantial
Student 156.8
Student 260.0
Student 347.1
Student 454.3
Student 556.4
Student 653.6
Student 754.3
Student 854.6
Mean3 (SD or 95% CI)54.6 (3.7)0.75 (0.69 to 0.81)
Adjusted AC20.64/Substantial
All observers mean (95% CI)0.79 (0.75 to 0.82)
Adjusted AC20.72/Substantial
Difference in means between two groups with 95% CI in brackets
 Researcher1–clinician24.0 (−3.7 to 11.7)
 Researcher1–student37.0 (4.4 to 9.5)
 Clinician2–student33.0 (−6.0 to 11.9)
  • Means of each group compared with paired t-tests. Classification based on Landis and Koch.14

  • *Critical value for all 24 observers=0.07, critical value for 8 observers=0.11.10