The key elements of the proposed procedure are
- the definition of the content domain of interest in relation to the learning objectives of the course
- the specification of an assessment framework
- the operationalization of the assessment framework with test items
- the standardized administration of the exam
- the scaling of gathered responses with item response theory models
- the setting of grade levels with standard-setting procedures
Empirical results obtained from six test cycles of a real university exam at the end of an introductory course on research methods in education show that this procedure can successfully be applied in a typical university-setting. It was possible to constitute a reliable and valid scale and maintain it across the six test cycles based on a common item nonequivalent group design. The comparison of the observed student competence distributions across the six years gave interesting insights that can be used to optimize the course.
This means an increase in the exam´s fairness, which, in turn, is a prerequisite for the validity of the interpretations ...
Andreas Frey, Christian Spoden, Sebastian Born: Construction of Psychometrically Sound Written University Exams. in: Psychological Test and Assessment Modeling, Vol. 62, 2020(4), pages 415-425
Literature:
Psychological Test and Assessment Modeling
Vol 62, 2020-4