Evaluating performance of environmental health assessments

<accesscontrol>Members of projects,,Workshop2008,,beneris,,Erac,,Heimtsa,,Hiwate,,Intarese</accesscontrol>

This is a manuscript about evaluating performance of environmental health assessments. It emphasizes the importance of proper identification and explication of assessment purpose against which (and only which) the performance of the assessment can be evaluated. It also suggests a set of general properties of good assessments that can be used as the performance criteria of any kind of assessments.

Background

• General assessment framework
• Societal context of assessments
• General purpose of assessments
  • to describe reality
  • fulfill specific needs
• Open Assessment
• How can performance of environmental health assessments be evaluated?

Results

• Properties of good assessments
  • quality of content
  • applicability
  • efficiency
• Relation of properties to information structure/content
• Evaluation process
  • a priori and/or a posteriori view
  • identification of purpose
  • evaluation of quality of content (uncertainty + relevance)
    • in principle reality, but in practice golden standard as reference point D↷
  • evaluation of applicability
  • evaluation of efficiency (effort expenditure)
  • overall performance
    • potential for effectiveness/effort given purpose
      • can be further evaluated retrospectively against realized effectiveness (possibly against redefined purpose)

Discussion

• Uncertainty as an aspect of performance
  • parameter uncertainty
  • model uncertainty
  • scenario uncertainty
• Data source reliability as an aspect of performance
• Performance exists only against a purpose
  • data about hypothesis

Conclusions

• There is more to assessment performance than just statistical uncertainty and data source reliability
• Overall performance of assessment can be evaluated systematically and explicitly
  • requires consistent information structure
  • a priori evaluation should be made an inherent part of assessment process