Evaluating assessment performance

Evaluating assessment performance is a lecture about the factors that constitute the performance, the goodness, of assessments and they can be evaluated within a single general framework. This page was converted into a lecture as there are other descriptive pages already existing about the same topic. The old content was archived and can be found here.

Scope

Purpose: To describe what assessments are about overall, what are the factors that constitute the overall performance of assessment, how they are interrelated, and how they can be evaluated.

Intended audience: Researchers (especially at doctoral student level) in any field of science (mainly natural, not social scientists).

Duration: 1 - 1.5 h

Definition

• Assessments must meet needs their of their use
• Assessments must strive for truth
• Both requirements must be met, which is not easy, but possible
• Assessment is a business of creating understanding about reality

In order to understand this lecture it is recommended to first acquaint oneself with the following lectures:

• Open assessment in research
• Assessments - science-based decision support
• Variables - interpretations of real-world phenomena
• Science necessitates collaboration

Result

• Introduction through analogy: mobile phone assessment (product) vs. mobile phone → product, production, use
• Performance related concepts in the context of assessments as science-based decision support: properties of good assessment
• Contemporary conventions: quality assurance/control and uncertainty assessment
• Assessments serve 2 masters: truth (science) and practical need (societal decision making, policy)
• Evaluation: points of reference and criteria,
• Properties of good assessments as a means of managing design and execution or evaluating past work
• properties of good assessments is not an orthogonal set
  • applicability depends on quality of content, efficiency depends on quality of content and applicability