Background There are challenges around the practicality of conventional standard setting methods for student assessment. Furthermore, accuracy of absolute methods of standard setting is difficult to achieve.
The aim was to determine which group of judges is most accurate at establishing the minimum level required to pass questions in order to ensure an appropriate standard (cut-scores), and how the Bloom’s level of each question affected the correlation of cut-scores to student performance.
Methods The modifications to the classical Angoff method where a group of judges convene and discuss cut-scores was that, in this study, the judges set cut-scores independently and did not receive the answers to the questions that they were assessing. Computer-based multiple choice and multiple response type questions were compiled, and allocated Bloom’s levels. Judges answered the questions, determined cut-scores and completed a questionnaire. Simple linear regression was used to determine whether number of years’ experience, proportion of time spent in small ruminant practice or specialisation in the field resulted in the most accurate comparison to student performance.
Results Individuals spending the greatest proportion of time in small ruminant practice demonstrated greater accuracy in determining cut-scores. The Bloom’s level assigned to each question was reflected on student performance.
Conclusion This study supports that the time spent in a particular discipline must be taken into consideration when selecting judges for establishing cut-scores, and that the cognitive level of each exam question be considered to improve accuracy.
- veterinary education
- small ruminants
- veterinary graduates
- modified Angoff
- predicting student performance
- Bloom's cognitive levels
Statistics from Altmetric.com
Funding This study was supported by Vice Chancellor's Academic Development Grant, University of Pretoria.
Competing interests None declared.
Patient consent for publication Not required.
Ethics approval This study was approved by the University of Pretoria’s Ethics Committee (V018-17).
Provenance and peer review Not commissioned; externally peer reviewed.
Data availability statement Data are available on reasonable request. The data obtained in this study are confidential in accordance with the informed consent agreed to by participants, and are stored by the researcher. These data may be made available on reasonable request in accordance with the University of Pretoria policy.
If you wish to reuse any or all of this article please use the link below which will take you to the Copyright Clearance Center’s RightsLink service. You will be able to get a quick price and instant permission to reuse the content in many different ways.